hey good morning good evening Mark how are you hi Dan good morning how’s it going hey

it’s going well man it’s Friday you know so I’m excited about that and we have a great guest coming up on the show today

so that’s exciting for me absolutely we are looking forward to speak to Ron yeah

how are you doing my friends uh not too bad it’s been a great week I know we’ve been working on a lot of chat

GPS stuff this week so that’s an exciting

which is exciting that’s a you know cool technology so we’re excited to integrate it into the platform and do other things

and of course talk about it so look for some upcoming articles and content from

us on that for sure I know some of the things we wanted to talk about that’s probably closely related because I

believe in the 2023 uh Trends report that we released is around you know generative Ai and GPT so we we released

a 2023 Trends report check it out Mark will put the I’m just putting it in there in the comments so you can go

ahead and download that for free it talks about five different trends that we’ve uncovered that we think will

continue on in 2023 so shout out to Danielle on the marketing team here who

helped put that together and then we also agreed yeah it was really good and then on the 26th I guess next

Thursday already Mark is a webinar around um consumer Trends and absolutely this

is with Steve Krauss who’s a professor at the University of San Francisco and John last who has a company with the

sports Leisure group and he does a barometer Steve and I kind of uh partner

on a survey around different Trends and we talk about some what what was in 2022

and then what was what’s coming up in 2023 so more from a consumer perspective but then we’ll also be covering some of

those market research Trends and so forth so um yeah that’s good I’ll just put a link

in there talking about chat GPT again oh nice okay

I knew it’s all about it I like it thank you Anu um awesome so those those are some

events and some materials that we have here at question Pro so make sure you check those out and I think Mark without

further Ado I know we have a lot to talk about with Ron here absolutely

Hey Ron welcome no all right on good morning nice to meet you both yeah

likewise likewise Ron so we’re kind of chatting a little bit before the show and I wanted

to get you know maybe tell people about your background and I know you have a stop in Asia which I want to talk about

a little bit so maybe give a brief description of your background and and then kind of where where you’re at now

yeah so uh yeah I worked for two major Banks doing market research I was their

insights director for two major Banks one in Utah and one up up in Seattle and

uh you know my job there was helping senior Executives make decisions right

and I loved doing that I was appeared to most of them and uh and it was pretty

challenging because you didn’t want to be wrong right I learned a thing or two about about doing research right so that

um so we so that it came out positively um one of the banks I was with when

there was a mortgage loan turned down downturn in 2008 when that collapse happened

um it uh Washington Mutual Bank where I was at I didn’t survive so Chase Bank got them and uh I remember that and I

left and went to Coca-Cola I became the inside structure for Coca-Cola in Asia

so I lived in Hong Kong uh for six months I lived in Thailand for six and a

half years and I traveled all throughout Asia helping China Japan Philippines

Vietnam Malaysia 14 different countries wow that’s amazing with their with their

research and strategy needs right so that was amazing you know um Ron Washington Mutual is a special

place in my heart because it was the first bank that I ever signed up for so really yeah

it was great well it lasted you know I know I love it I loved it I was like oh

I got my first paycheck and I’m like well what do I do with this or I’m also we need to opening an account so I think took it down there and got it yeah it

was a good thing a few poor choices along the way but a great Bang Yeah amazing amazing so Coca-Cola so you

moved to Asia for this obviously you mentioned you know Hong Kong and Thailand what are some interesting stories that you can share I’m kind of

I’m curious about that do you know yeah I am an interesting thing is I was fortunate

to go to Myanmar I don’t know if you’ve heard of Myanmar Burma before yeah but um it was closed to all U.S

business um until Obama got in and then relationships were opened and modified

and I was one of the very first people to go in for for a business point of view to figure out how Coca-Cola could

enter in that country and I spent three weeks kind of going to length and brought the Myanmar getting to know the

people getting to understand what the company ought to keep in mind and sharing our thoughts is the best way to

go forward wow that’s amazing really something what a unique experience too that’s that’s

awesome yeah cool it was actually I was in a lot of people’s homes went out to dinner with

people there they were so kind you know it was a wonderful experience and yeah and they were not

um they were not afraid to you know just welcome in a new opportunity there

either it was it was a very fascinating thing

and Mark you were asking if Ron had ever been to India is that right yeah have you been to India in yeah several times

several times so uh but both in New Delhi where I spent some time but also

down in Mumbai and uh and uh also I spent some time in the poorest parts of

India to figure out how expansion could realistically be how we could help in in

those communities so it was a fascinating thing to spend some time in India yeah that’s amazing yeah

yeah really cool and then so from Coca-Cola you had a you started up

mobile digital insights is that right right so I always wanted my own research company and uh and I started that and uh

mobile I partnered with a with a company Mobile digital insights it was in Australia so I did the North and South

America business and uh and uh the Australia business continued and I so I

did that for five years enjoyed my great relationships with them but uh along the

way while we were doing work with mobile digital insights we’re working with Coca-Cola KFC T-Mobile some very nice

companies like we had a great Client List and um along the way a city manager of a

city in Texas approached me and asked me if I would help him with his uh annual citizen survey

and I had never done an annual citizen survey so I looked at what he had um and uh it was a it was a an

interesting survey let me just say it was probably about 20 minutes long covered a lot of ground

and I could tell from the get-go it was almost impossible to make good decisions with just a lot of information tons of

information right as a matter of fact so much you couldn’t even figure out what to do wow and so I I just said well I

can help you and so we thoughtfully redesigned it as a matter of fact I got two or three other City

surveys just to get my mind around what what is happening in the industry and just crafted a totally unique approach

to help him achieve what he wanted to achieve and get what he wanted to get he loved it I loved it and actually when I

finished I said you know I like this more than helping businesses like I don’t mind helping businesses but I

enjoyed it more helping these communities right and these city leaders and it was really fulfilling for me and

so I said that’s what I want to do for a business so I Unwound myself from the other company and started up on point

insights and now my focus is cities wow and that was that was recently then Ron

or it’s been I’ve been thinking about it for a while but I really just started last year oh interesting what what city

was it in Texas can you share Bernie Texas Bernie Texas okay I’m in Round Rock Texas but

b-o-e-r-n-e Bernie Texas it’s a German Town and and I uh and I’m doing the

second round for them they loved it and we’re doing the second time so interesting actually this is total side

note but there’s a lot of German influence in Texas so there’s is a lot of these small German towns which are great to go to because they have good

German food and good German beer too so as a side note but probably in Bernie too probably have

to check that out um awesome so what were some of the you know can you give some context around the challenges that we’re facing that

you know kind of Life yeah you know you know what I did I pulled together a couple of slides and I’m yeah it’ll make

it easier for me to talk to and maybe even easier to even visualize so I’m going to share my screen screen share

screen share share okay so I’m sharing my screen and now I’m going to go to PowerPoint and I’m gonna do uh I’m gonna

share can you see that okay yeah this is this is great okay so let me give you an

example of what it is so um so his survey and it’s not unlike many surveys so he had he had in the

survey there were a variety of question types how much do you agree or disagree with the following and they had a list

right or how satisfied I’d argue with the following and they had a list and

sometimes they had like ten in one and then another one and you know they just had all these lists and

how would you rate the service provided by some group right and you had pretty one of the mill questions kind of

like you see it really there were a lot of these and they had other things in there as well but they had a lot of them and and so the first thing I noticed is

we’ve got multiple scales there how do you compare the agree disagree satisfied dissatisfied good very good right and so

there was multiple scales and and really the survey itself was unified around the

question type and not that not the topic or theme so much right so yeah it would

kind of Orient around that and also we’ve I found that the same question it was like phrased slightly differently

but it was in like all three places sometimes you know where where you know how are we doing about this and and then

how satisfied are you with that with this and then how would you rate the service of this you know and but it was

essentially the same question sometimes that’s two or three ways right and then

oftentimes when we got the results you’d say okay where what like you’d show the top two box or whatever and everything

was top two box with minor variations right yeah and you couldn’t really tell

and in the end there were 54 questions that people had to answer and so it’s a pretty long and frustrating survey and

and this was like this is like the core of the survey maybe this took about 10 to 15 minutes long then they had other

open-ended questions and so forth to try to follow up and get their mind around it and in the end you had 20 minutes of

drudgery for these poor citizens right right that’s true I mean these long

Matrix questions and things that are really repetitive because like they were they work but they don’t you know but

but but that was the challenge and so he had these he spent like this this city manager he spent he said 120 hours

analyzing the data and in the end he had a beautiful 100 you know 100 page deck

or so that he put together but I still didn’t know where they were going right right so they just had pages and pages

of information and in insights but no Clarity of Direction and so I I I took this and when I did it

I I the challenge was when we were looked at it like all right how do we get to the vital few

things to do how do we know what they are and there’s this cities have like everybody you know they have all of

these constituents who want something that you know they they want a ballpark

they want pickleball courts they want Parks they want roads they want infrastructure they want you know they

have all of these things how do we how do we look at that and still how do we keep the survey short

like that was my challenge yeah right as I thought about it because I like a

10-minute survey um or no longer right I like to keep it around 10 minutes and and they were 20.

and so so the approach I took was I call

it the concerns of the mini framework and so so what I did was I took all of

those different comments and I broke them down or all the different questions and I broke them down into themes theme

one theme two theme three three and four theme five so I broke them down into themes and what were they trying to get

at in these themes and and and I tried to make sure a theme had no more than 10

uh questions in it um and if there were more than 10 we would kind of group them together or we

make sure that the that we really only talking about a single topic and and we uh without problem we could

get to the themes and and and so then we very simply asked people at the

beginning we said this is going to be slightly repetitive but what we’d like you to do is look at this list and tell

us which two are most important to you and then we go into the next and they

just they would pick their top two out of ten or nine or eight or whatever they had in that group of themes and then

which two are most important to you and the next theme and the next one and the next one they get they would get through

them in about is about 30 seconds a piece you know they’re going through click click they’re picking their thing

by the time they got to the end they had selected 10 items out of 50. that were

most important to them and then we said all right perfect now that you’ve picked your 10 we’re going

to show you the 10 that you picked and now we’d like to have you rate them so

people only rate the things that were most important to them number one make

sense and they rate those 10. and so so I mean the the issue is is now in this

case we have one scale for importance all the way through and we really have one scale for performance all the way

through the people have rated what’s most important to them the survey is half as long and uh and it works really great and the

way it works I’m going to just show you a slide here really briefly but yeah it

works one question one question so is there a reason you stuck with two or was it just because

I have tried three and I’ve even tried three or four and I even tried it a couple of different ways I had a two

most important two least important when I when I did it I would the most important least important and so so I’ve

tried it variety of ways I have tried three I’ve found that whatever the

number is that you have if you divide by four that’s a good number to have right

so like if you have 16 and you divide by four you could have pick the top four if

you have 12 and you divide by 4 you could have three but if you have ten or fewer two works really about the best

yeah and the reasons why is like my first choice and my second choice

they’re pretty important my third choice is important but then everybody’s third choice starts watering things down and

you begin to get a lot of this uh if you get less differentiation yeah that makes

sense that makes sense yeah and then so you let’s say you have five here and then each one they pick their two most

important they roll into the ten do you have them rate that ten as well or so then all they have done is they’ve rated

the two most important we we could rate the top ten but in this case we get the

the very good the very poor and you know you really have everybody’s talk you actually have the greatest top 10 out of

50. you know out of 50 you have their time now there’s some benefits in it and and

and and I’ll show you the benefits the challenge is is it does need a little bit more sample size

and um um I’ve got somebody who just gave me a call I’ll tell them I’ll call him back

later all right but um but you do need a little bit more sample size if you’re going to slice and dice down to details

right because the good the very good the very poor ratings are

only among those who picked it as being very important I got it got it and then

um you know a new pad one question here so you have a skill for importance but what’s the scale for a performance can

you explain that yeah in in this particular case it could be any scale you want but we use very good good okay

poor very poor got it so how are we doing on these

attributes is the question we could say how satisfied with are you with the way we’re doing this you know

and the only reason I picked that is because it’s short and simple right yeah

and and and and uh but but there’s no reason why you couldn’t have it be a

strongly agree or a very satisfied I happen to choose very good to very poor

and it worked perfectly yeah that makes sense so it’s kind of like the important satisfaction but you have importance and

performance those two things makes sense awesome so you have this framework and then you roll it into you’re going to

show another thing that you roll up yeah here’s an example of the way it looks so on the left what you have is what’s most

important right the two that are most important in this group this is a within

a group and so you get the two that are most important they they rate them and

you could see 51 said city parks Pavilions benches playground that was the most important to them out of this

category this theme but only five percent said the community center was the most important thing to

them now the five percent at the bottom still were asked about the community center how are we doing please rate US

very good to good to verb to poor very poor but in general they’re doing pretty good right I mean the community center

so they’re people who consider it very important and hey it’s not a problem right and the library 25 said hey the

library is the most important to me in this list you got that and you know the library is doing pretty good you know

but the walking trails was second of in importance and it looks like there’s some room to improve right so here a

city can very quickly see then like where to focus and and and it’s

true some people want some dog parks and maybe it’s like it’s like not even a bad idea to do dog parks right I mean you

could you just realize it’s not the most important thing right and for the majority of the people and so it’s it’s

a niche thing and and by the way if you’ve addressed the things up in the top or you feel pretty good about it

move on down to the other things that are low of lower importance and so so

just you know in a court in a really quick summary I mean that is the core methodology

that’s interesting I mean things like walking trails are probably you know really important some of those from like

a city perspective might be harder to implement right but you may want to move on down and say hey we we can get this you know the food

truck thing is easier whatever it might be but it definitely gives it definitely gives clear sort of view into what you

need to work on and what might be the correct and it can do this in in Safety and

Security they can do this in like infrastructure and roads they can do this for each of the various topics that

we have and then within them they can know what’s going well and and where

there’s room to improve and so it’s a I mean the beauty of it is is in truth I

don’t have to tell them where to focus do you know how when you get to the end you say you know here’s where you should

focus well in truth I can tell them that but they can tell me where they should focus right and so oftentimes when I

present this to a city I give them the results and I have them walk me through the findings and tell me what they think

they are it ought to be done right because it’s so obvious and intuitive

that they don’t have to wonder and so they they’re 120 hours of work that that

the the manager spent in Bernie trying to figure out where they should go he

can actually not try to digest the data he can try to figure out now the solutions right right one more spend

more time on what to do with the data than just yeah he spends like no time on now the analysis is obvious and done

that’s interesting because you talk a lot about or I think the industry talks a lot about you know storytelling with

data but this kind of tells the story in itself so it takes a lot of the work out yeah I will tell you it’s it’s very

interesting I was I’ve been speaking with city leaders across the country and and one of them I was talking to

recently and here I’m going to stop sharing my screen now I’m sure yeah and then

um um I’m gonna stop sharing here okay okay so what I’m gonna do what I found I

was speaking with city leaders across the country and um a couple of them said I said hey do you do citizen surveys and

they go yeah we do them and I said how’s it working for you and he goes well you need a PhD in statistics to understand

it and I said really you know you laugh and I said you know what

that’s not the problem the problem is is the person on the

other side doesn’t know to tell you what to do either they’re clueless they have no idea the survey design is not

intuitive intuitive enough that either of you can figure out the way to go and so it’s not like you need a PhD in

statistics you just have a failure of design right that’s true to make it to

make it intuitive and then you know I know Ron you use um

I’ll give a quick plug here you don’t know you use question Pro for this yeah how’s this question probably able to

help you out where you know let me tell you Dan when we when I looked uh I knew what I needed to do to accomplish and I

had not chosen a survey supplier for this right so uh I cast a broad net and

looked at many companies um many well-known companies uh in the

top a lot of them a lot of them claim they could do it and then I would do a trial and they couldn’t do it easily it

was really difficult so and then a lot of them claim they could do it and frankly they couldn’t because

one of the things you have to do is you have to be able to take questions that were asked five before and you have to

be able to then put all 10 of them in front of somebody to rate them yep right

and so it’s not a it’s not a simple extraction from one a presentation of

the other it’s an extraction from five and a presentation and one yep so it’s a from many to one and and uh and that’s

that’s not easy to do so when I created in my first city I didn’t use question Pro I used another and we got it done

but the amount of work I had to do to make it work was a job in and of itself

and and and then uh a question Pro on the other hand look it wasn’t 100 easy

for question Pro either and I know question Pro is rewriting and creating Solutions so that this particular need

can be addressed easier but I was able to do it nonetheless I got some great coaching and within

um it took me a day to write the code yeah to write the code and to do all the

extractions so that it would work it took me a while but then it like worked flawlessly for the next city the next

city the next city because I had already put the work into it but I know question Pro is working on it so that that need

doesn’t exist like it’s just a simple distraction yeah we are yeah yeah actually I just talked about the

development team about it this week so I know we’re working on it yeah yeah that’s awesome

yeah but but so question Pro the number one question Pro was able to do it they were able to do it so that I could

do it repeatably easily and then they were able to do it so that and they were

able to modify to even accommodate this kind of a need for others in the future so it’s not so difficult for anybody

right it makes sense that’s awesome um Ron hey thanks so much for sharing

you know your groundbreaking model that you know they’ve told me like hey you got to get run on the show he has he has

this model so I’m excited I was curious it is unique

yeah it’s super unique and interesting and what a way to sort of make make

sense of the data like I think you did a phenomenal job there yeah I’m telling you I’m telling you it

is it is city leaders have just simply loved it yeah like they just loved it

and because it doesn’t take anything a good set of eyes and a little bit of

judgment to know what to do you know what I mean it’s not like yeah it and and I I it doesn’t this doesn’t apply to

everything but there are certain situations where it probably is the best approach you know and a very good

approach and you know when when there is a large number of challenges that you

have to choose between and sort through this works yeah I’m sure you know when

you go from the many to end of the few and and and and and I was speaking with a city leader and and the person just

said what what makes your model unique and I said well it focuses on the needs of the many

um it’s a perfect way to explain it I mean it really is and it makes it make sense it’s easy yeah and somebody said

well what are the needs of the many like and then they’re kind of mocking because you know in society in today’s society uh government is kind of like

you know it’s it’s tough because government’s going all kinds of different directions and what this does is this helps city leaders not lose

sight of the need what’s important of everyone that are so common you know and

and not get wrapped up around the things that are just kind of one-offs and they’re very important like all of them

are important but some you don’t want to lose sight of the needs of the many right and I think you could have a like

a vocal minority too right that you’re like oh we need to tackle this but then you do the serving it’s like actually

that’s towards the bottom most people don’t even want that so that’s that’s interesting that’s basically what happens yeah exactly what happened right

I bet and it just allows it allows city leaders cover to like either they can get to that but yeah but only after they

focus and assure that many are thoughtfully met you know and it keeps

them from being pulled into directions that are that are you know meaningful but maybe not as meaningful right now

right yeah so one one I think final question I had a

more around response rates so the traditional model that cities were doing with you know let’s say the 54 questions

20 minutes and you take it down and you go 10 minutes I’m guessing the response rate is obviously higher right but is

that obviously I don’t you know I I I’ll have to confess I don’t know but I can’t say yes but let me let me tell you where

we come from so when we had the 20 like the first time I did Bernie the first

city we got around 800 responses which was pretty good for the city yeah you

know which is pretty good the second time we did it we’re over 1300 right now wow right and so citizens citizens who

took it the first time and took it the second time became more engaged the second time yeah and my own City I did

it in my own City where I live in Utah uh Elkridge Utah it’s a town of four thousand right this little town but I

did it for them so that I could do a proof of concept before I wrote live with Bernie so um I did it for them

first time around we got 500 which we were impressed so you know we’re getting about a 20 to

30 increase um the second time around over the first and I think part of it is that right

that that the willingness to participate because the survey is short and it’s also intuitive and pretty easy to take

um makes for a difference yeah I think so I think so especially if word gets out and I think if if city leaders can

touch it back like and we came up with this project or we did this because this is what they took that survey right and

then maybe that doesn’t catch on but in my head I can see that no no I’m with you hey so Dan if you go to Saginaw

Texas or Coppell Texas okay or you’ll find or or West University Place Texas

you’ll find on their websites the results of the survey and it’s there

for everyone to say saying thank you for participating in the survey here here it is and and and the way we present it to

you it’s even more intuitive for the city leaders I I’m not going into detail on that because it’s another topic yeah

but it’s like so intuitive to go and look at the results and to see what it is and they make it available for

everyone to see that’s amazing awesome awesome Hey Ron thanks for coming on

today where can people get in touch with you if they want to learn more about it is LinkedIn a good way or I should have

given the name of the company but it’s on point insights so um how do I do it uh you know what uh no I’ll just it’s

all spell it out so on point is o-n-p-o-i-n-t-e okay one point with an e

on the end Dash insights.com that’s my website and then if you want to call me

it’s Ron at onpoint-insights.com on point with an e

great well that’s awesome well Mark let’s update the description too so we can yes

awesome yeah Hey Ron thanks so much you know the quality I’m sorry I did I said

qualtrics and I should have said that yeah that’s fine the question pro team has been amazing

to work with as opposed to others so um you know and I want to thank you for your great collaboration absolutely

anytime um I definitely want to have you back on because I have more questions but we’ll have to do a part two at some point yeah

because I have a lot to share awesome sounds great I love that Hey Ron thanks for coming on really appreciate it

um and have a good day take care bye-bye see you guys

thanks