How to write a survey?

OK, so you figured out you need to conduct a survey. What’s next?

Everyone’s gut reaction is, let’s start writing questions. But if you want to do it right this is exactly what you should not do. In fact, resist all urges to even think about what questions to ask. To write a better survey, the first question you need to answer is:

1. what am I interested in finding out?

It is very important that you do not phrase the answer to this in a survey question. You never really cared how people respond to any question anyway, right? You care about what is behind those responses. Let’s say, you want to know if a customer likes a product. Maybe you want to know if they are likely to refer you, as a service provider, to others. Ask yourself, what are the relevant demographic information needed for your study. Make an exhaustive list of what you want to find out.

Now you write questions? No! You still should not. Rather, ask yourself…

2. what will you do with this information?

You have to think about this question in two different ways, two different sequential steps.

a. How will I analyze this data. Develop an analytical strategy. Will I look at (or present) a histogram? Do I want to see any association between two of the constructs defined in point 1? Are older or younger people more likely to refer my services to others? Men or women? Compile an exhaustive list of such questions you have, that you want to get out of the data. Once you asked your questions, come up with an analytical strategy. Do I just want some descriptive information (like a cross-tabulation) all the way to needing to develop an instrumental variable regression model to causally ascertain the relationship between a key independent and the dependent variable? For the latter you may also need to find three or four plausible instruments. Have an analytical strategy in mind. It could be as simple as calculating the mean, it could be a simple inferential statistic like a correlation or a two sample t-test or it could be something complex. Just make sure you have a preliminary analytical strategy.

b. Know what you will do with the information. Devise action strategies. If I see that young people do not refer my service to friends, I will develop a marketing strategy that will nudge young people to do this. Maybe you are not the person taking action on the survey, then devise a recommendation strategy. If you work for a client (even if it is in-house) push them hard on devising this strategy before you start the survey. The better they know what they want to do with the data, the better chance you have a writing a useful survey for them.

To aid steps 1 and 2 (which you may need to go back and forth on a bit) it is a good idea to draw things out. It is OK to go back and forth between your current and previous steps, but don’t go beyond that.

3. Take the constructs you identified as crucial and figure out how best to operationalize them. Chances are, this is the stage where the constructs (if you are following these steps closely, that are drawn up in a web of relationships of interests) are going to start turning into survey questions. It is OK to ask multiple survey questions trying to tap one construct. Beware (and if need be, modify) the analytical strategy developed in step 2 as you start operationalizing. Maybe you thought you will look at a correlation, but it turns out a simple yes-no question is the best way to ask about something. Then you will only have a dichotomy and not a continuous variable as called for by a correlation. So, you may need to adjust your analytical strategy. At this stage, don’t go back to step 1 anymore.

Follow conventional rules of questionnaire design. Make sure you are asking questions (not just throwing words at the respondent. “Gender: ” should be “What is your gender”? The survey process is a conversation. Don’t break the basic conversational rules. Make sure the response categories you offer are unique, mutually exclusive and they answer the question you ask. Label all your response categories and no need to throw numbers you will use in the analysis at people. Unless you are some survey researcher or quantitative social scientist (which you probably are, or slowly becoming if you got this far), it is wholly unnatural to map a conversation on to some numeric space, so don’t make people do it. Also, don’t even bother them with your numeric mapping. And – very important – make sure the response categories actually answer the question. If the question starts with “how many”, the answer is never “strongly agree” or “disagree”.

Remember that bipolar scales should be no wider than 7 points (11 for experts – but good luck labeling all of them…) and unipolar scales no wider than 5 points (7 for experts). Don’t let your respondent just run through tables of questions with the same responses. They will lose attention. Better if you write question specific response categories.

Write at a grade level that is around 3-5 years lower than the lower end of your population. Don’t use big words, homonyms, heteronyms or jargon that may not be understood by the respondent. Ask one question at a time (the words and + or are usually red flags in survey questions). You can offer a don’t know option, just remember, it encourages people to not engage with the survey, not to think about the survey. (And men, on average are less likely to admit not knowing anything anyway, so even if your goal is to find out of people don’t know something, just know that your results will be biased no matter what, so why bother.) This is hardly an exhaustive list. But there are a million more pointers and also great survey question writing tutorials online. Read through a few. What I see less of is tutorials demonstrating this broader process that looks beyond the question writing and, IMHO, is absolutely necessary to acquire quality responses that you can effectively use.

Finally, please remember that most people hate surveys. This process ensures that you only ask what is necessary and what you know you need and know what to do with. The longer a survey is, the worse the data quality will be. Off the bat, fewer people will take a survey that seems long to them (and it is a good idea to tell people anyway how long the survey will take to ensure they have enough time to do it when they do end up taking it – some people may never take a survey as they don’t know if they will have time to do so, unless you give them a ballpark estimate of how long). People’s attention spans are more and more limited in today’s day and age. After 10 minutes, you can forget about them paying much attention which will be at the expense of data quality. This process ensures no unnecessary questions are asked.

When you start a survey by writing questions, you become fond of those questions and more likely to ask them (or hesitate on cutting them later). This is why it is especially important to first know what research question you need answered and only then, start designing the survey questions that will help you do it.

Of course, there will always be that stakeholder who comes and says, we also should ask question XYZ… and sometimes they have good ideas with obvious implications. But most of the time, this is not the case. The best weapon against such a proposal is the demonstration of the thoughtful development as described above. You come back to them demonstrating how and why all questions are in this survey and asking them, now with this in mind, why do you want to ask that too? They will either improve your design on the spot or back down. Either way, it is a win-win.

Headphone Geek Out

It has been a while. I need to get back into the habit of writing. Right now, I am writing grant proposals and there is nothing as soul killing as those. I hate it. Maybe this will be therapeutic.

So I threw away a headphone or two lately. They were dying. So it was time, once again, for that semi-regular activity when I contact Juraj Medzihorsky, who, in addition to knowing everything there is to know about music in Budapest, he also knows everything there is to know about sound in general.  From a fun scraping exercise, once he scraped some headphone review site and came up with the following scatter plot. (Y Axis is sound quality, X axis is log price).

It does not take a stats genius to figure out which headphone one should buy. (So I did, and since, I consult Juraj on headphone purchases.) I don’t have this headphone anymore. It had no mic, no button so I actually did not like it for day-to-day use.  It did sound amazing. (OK, Szása called me out on Facebook for this. It was the Monoprice 108320. And yes, I totally wish I would have kept it so I could do a comparison now.)

I had a horrible time with headphones around then. Klipsch S2/S4 were dying every 2-4 months. Etymotic lasted 6 months. There were others as well (maybe a very specific cheap Samsung which was OK), but the first true love ended up being the Xiaomi Piston 2. Those things were cheap, durable and they sounded great. There was only one thing about them that bothered me. They got discontinued.

Piston 3 sounded like shit IMHO. The 4th generation Xiaomi Piston Dual Drivers sounded fine but they did not have the durability of the Piston 2. In fact, the one I threw out recently was the last one of these I had.

So it was time for new headphones. There was buzz around a Sony and they were cheap outgoing models. Meh. I guess they work but there was nothing about them I really liked. Sound was OK, nothing special. Feel was weird. Flat cable didn’t tangle so I kept them as the back up in the bag, in case I forget my daily drivers. But when I found myself digging on the bottom of a box for something to listen to podcasts/music with, I knew it was time to make new purchases.

Talk to Juraj. Industry favorite is apparently the KZ-ATE. OK. Lets grab two. I also managed to find a Piston 2. Made me so happy. And as I tried it out today, it is certainly not a fake. (Yeah, it is that bad. They are counterfeiting the cheap headphones in China. So everyone BEWARE!)

(This was the worst rambling way too long intro I ever wrote.)

Well, this was interesting. I put on the < 20 EUR KZ-ATE and I knew I was up against something very-very serious.  So it was time to do some more in depth testing. I did not use any special equipment. Just my Google Pixel Phone and Amazon Music. The test songs were Tom Russel’s Man from God Knows Where (my favorite test song) and Chris Thile’s I’m Nowhere and You’re Everything. This covers the style of music I listen to where I most care about quality. These are songs I know very well. They are well recorded.

The KZ-ATE had incredible clarity and detail. Despite the stupid size (don’t worry, they are not as uncomfortable as they look) it was not bassy. I figured the rationale behind the size was to fit a big driver in there. In fact, it was surprisingly not bassy. With the clarity I was up against, I expected more bass. The roll off on the low-end was very smooth though. The top was weird. The resolution was incredible. I could hear every little detail in the music, but it sounded off. The acoustic guitar overtones especially were simply incorrect. That is not what a guitar sounds like. Mandolin was ditto. Thile’s vocals were also off. It was such a strange feeling to hear all the details and hear it incorrectly.

For good measure I did a listening test with the Piston. Nowhere near the resolution. The bass was also not there. (OK, maybe a bit more there, but definitely not a big difference.)  The sound was not nearly as sooth. Resolution was not there at all. But the guitar and mandolin (not to mention Thile’s voice) sounded right. And the sound stage was open and musical.

There are so many terms in the audiophile vocabulary and I don’t know what most of these mean (but I swear, neither do anyone else – this often reminds me of the fruits people list off when tasting wine). But I felt that the KZ-ATE has incredible sound within that middle range. Once we got to the top and bottom, it was not correct. I know that many people are in love with their single driver speakers. I assume this is the kind of sound those has. Missing top and missing bass. But it is very musical, easy to listen to and the detail will not bother your ear/brain. (I am guessing here. Never heard such a system.) But I still prefer the Piston. (Should hunt for more old stock I guess and pray they are not fake.)

I put the KZ-ATE through some burn in. Have not tested since. If it turns out I didn’t do them justice, I will update.  I do wonder which of the two causes more listening fatigue. I know my home system does quite quickly and it is probably because the incredible resolution and the sheer high-end coming out of the horns. (I am sure the cross-over is also not helping, though that has been updated since.) I don’t know if it is the resolution of the KZ or the high-end clarity of the Piston 2 that would tire the ears/brains more. But I do know which of the two I prefer.

This said, at the price I paid, the KZ-ATE is an incredible headphone. It does not happen often that I hear a headphone and I schedule a dedicated listening session to figure out what is going on. I didn’t even do this for the Piston 2. Only that outlier dot on the figure got this treatment before and that was a biased pre-conception. The Sony never got the honors. I knew from the get go that they were nothing to write home about. So, at this price, they are recommended despite al the “bad” things I said about them above. They are great headphones.

If anyone is interested, I got them here.

Bulletproof Coffee

A few weeks ago as a van full of music geeks were heading back from the Chris Thile – Brad Mehldau duet concert in Vienna, I found out that I am also sitting with some serious food geeks which devolved into quite a conversation with one revelation (at least for me).

From the colleague who (objectively) offers the most popular course at the ECPR methods school (who also turned out to be a serious food geek), I already knew about the importance of medium chain triglycerides (MCT – translation: fats) for anyone (like me) on a low carb diet. It is an important substitute of carbohydrates in providing quickly accessible energy for people whose bodies are already used to burning fat for energy. Best (natural) sources of MCTs: coconut oil, palm oil and cheeses.

So apparently this is a thing. One spoon full of coconut oil, one soon full of butter and coffee. (And it is not as bad as it sounds. I add a sweetener and top it off with some milk.)

So since my body adjusted to the low carb diet, I really have not been eating lunch. Even before the diet, when I had a nice big egg breakfast, I was always OK until an early dinner. (But when I ran out of the house without a breakfast and grabbed a pastry on the way with a yoghurt drink, I was already starving by lunch.) But this may just become my substitute for the run out of the house quick meal (and plan on, probably, an egg based lunch). Last week’s experiment did not work so well, but I think the 4 hours of sleep had more to do with that than the meals.

Give it a shot… but until then, enjoy Chris Thile and Brad Mehldau (though I prefer this instrumental cover of theirs from the late Elliott Smith… and also Don’t Think Twice and Marcie for which I did not find good live recordings). But this is the most representative of their work.

Innovation in music is generally bad, but…

I think it is fair to say that just about every technological innovation in music since about the time I was born left the world being a worse place (be it the CD player, mp3s, mp3 players, mp3 compatible CD players… transistor amps, digitized techno, digitized anything, everything in the 80’s and etc.). Exceptions are few and far between, but today I came across a big one.

This trinket is certainly not for me. But I appreciate the fact that it is a modernized classical coil based guitar pickup design with the string separation nobody ever cared to develop. And the possibilities for musical creativity is are quite amazing starting with the idea carried by its predecessor allowing the insertion of a (low) octave effect on the lower two strings. An ability to put effects on certain strings independent of each other, and to do so with the same analog effect pedals running into the same tube amps that made music (before I was born) what it is, is kind of amazing. For someone to approach the same problem that has already been addressed (with disastrous results) by Roland midi pickups in the 80’s in a truly analog and sonically conservative way is true genius. I cannot wait to hear what people do with it. Until then, if you are a more creative guitarist than me, get one of these! But definitely see the demo.

How you can never be smart

Apparently, when I was a toddler (not like I remember, but I am often told), my parents were also not morning people. So to prevent me from waking them up, they left sufficient entertainment and some food (chocolate milk) for me in the crib at night before they went to bed. This way I would not stay hungry and entertain myself in the morning while they crawled out of bed.

Some 38 or so years later, Stela and I are also not morning people. So we decided to try this same approach to manage our little monster. Been doing this for a week or two. One day she didn’t drink the bottle. Otherwise all seemed cool.

Yesterday we noticed that she found the bottle shortly after we left it, a little after midnight. OK. Must have been a fluke. Today, we noticed that she woke up at midnight and was frantically looking for the bottle of milk.

Try to be smart in this world and a 16 month old will outsmart you. Now we have to figure out how to decondition her from waking up at midnight in search of food. We’ll never know but I am fairly sure she has been drinking the bottle of milk just minutes after we left it for a week or two now.

Best music gear review ever by ItchyHead from Memphis, TN

I was monitoring the Cyber Monday deals for music gear. Musician’s Friend put up the Hohner Roadhouse Blues Harmonicas – 5-Pack (Keys of G, A, C, D, and E) in Custom Case for sale. They were dirt cheap, so I glanced at the reviews. (And, as expected, those sucked badly.) But in there I found a true gem of a music gear review titled: These caused me to get forcibly removed from a blues jam. I am sharing.

I am twenty years on the harmonica circuit now, with such blues experience that you would not believe it. But my last harmonica set was destroyed in a flash flood and I needed a replacement quick. I ordered this pack of harmonicas and took it down to Blind Willie’s to sit in on a blues jam. No sooner did I blow the first note when I knew something was wrong. I have struggled all of my life with horrible head lice problems and something in these harmonicas is like a magnet for these wily lice-bugs. They came pouring down my face and swarmed into the harmonica as I tried to add some panache to “Gimme Three Steps”. I inhaled sharply and several lice bugs flew down my windpipe, causing me great discomfort and horrifying the patrons and the band members. Blind Willie is a superstitious old fool and all of this convinced him that I was the first of a new seven plagues of Egypt so he threw me and my now lice-ridden harmonicas out into the street. I was almost hit by a Geo Metro.

BOTTOM LINE No, I would not recommend this to a friend

Last trial run of the MSEM book, in Bamberg

Last summer I taught Multilevel Structural Equation Modeling at ECPR’s Summer School in Methods and Techniques. This summer, my coauthor, Bruno Castanho Silva did the same course. Today, we have a complete draft of the MSEM book (also co-authored by Constantin Manuel Bosancianu). For the last time, before we send it to the publisher (SAGE, QASS), we get to use it in action. I need to thank the University of Bamberg (with special thanks to Thomas Saalfeld) for inviting me to do this workshop on MSEM and giving us the opportunity to see how the book works in action and get feedback on our full draft.

How I came to love Strats

The sound, I always loved. My dad is a Strat player. And I had many Strats and similar guitars (including a Fender blue floral MIJ reissue when I lived in Nebraska). But those guitars and I could never agree with each other.

I was always more of an acoustic player. One thing about acoustic guitars is that they have no (like zero) crap on the top to be (potentially) in the way of playing. A Strat is a delicate machine where you have to strum paying attention to the knobs, switches and even the middle pickup, which is sometimes (or in my case, often) in the way.

The middle pickup issue is the easiest to live with. I always preferred guitars without a middle pickup (none of which would ever sound like a Strat), but as long as they leave me some space (and Strats do) I could work with it. A two humbucker with pickup frames and middle pickup, not so much…

The pickup switch placement is my second vice. A looser strum and I change from neck to bridge pickup. (And that is a change people notice even if you switch back quick…) But since, my playing is more refined. I can work it.

But this third one, I cannot deal with. The damn volume control. Apparently, the place I rest (or have, or whatever) my finger when strumming is right there at the volume control. I understand that having the volume control handy like that comes with great benefits, some tricks you cannot do, say, on a Gibson or a Tele, but those tricks are not me. And the natural movement of my hand while playing, what does it do? It turns the volume down. Basically, I cannot get through a quarter of a song, no matter how much I try, without turning the volume down, and usually all the way down. (And that change people really notice.)

But I love the Strat sound. I have been thinking about custom builds that overcome the issue. Maybe one that leaves off the volume control, puts it at the second knob leaving the third as overall tone could work. But it never looked right whenever I saw one of these set ups. (See above.) Or just pull a Tom DeLonge (as seen on his signature Fender Strat) and have no tone.

Still, the only Fender Strat I have seen that pulled off the limited knob set up was the pink Hello Kitty Squier with the slightly modified pickguard. But that will not get me closer to a Strat tone either (and neither would the Tom DeLonge) .

Enter a trip I once took to Stageshop, the best and really only worthwhile guitar store in Hungary, where they handed me a beat up looking Fano guitar. (By far the best guitar I have ever played, with a price to match at 7 digits in HUF – granted it started with 1 and ended with six 0s, but still – 3250 EUR, 3800 USD) That white Alt-de-Factor SP6 (actual one pictured below) had, what I learned to be called, a ToneStyler instead of a tone pot. (Though it was not the reason it was the best guitar I ever played. Not even close.) Here’s how the ToneStyler works. It is (in the case of the Fano) a 10 step rotary switch and every one of them sends the signal through a different value capacitor. The tone control rolls in a mix of the capacitor into the signal (I am sure I am screwing the terminology up – electricity is less my thing than sound). With the ToneStyler here, you can select 10 different sounds.

The ToneStyler is simply a better tone pot, in my opinion. Unless you want to use it for smooth sound effects, I think it is superior in sound and control in every way to a traditional tone knob. But they have one additional property. They don’t simply turn like any other volume or tone pot. It requires a bit of effort on the part of the player to switch between the positions.

So I got to thinking. What if I just replaced the volume pot in a Strat with a ToneStyler. The end of the story is that this worked like a charm. The rotary switch does not change just by gentle accidental touches to the pot making my main Strat problem a thing of the past. Sure, if I hand my guitar to someone, they are weirded out by the fact that the volume pot is a switch, the middle pot (well, maybe I should not get into it in detail, but it is called a Dan Armstrong wiring – look it up if you are interested) and the last pot becomes the volume down where I am use to having my volume on the Tele, far-far away from everything I could accidentally hit. Except, now I can also have that Strat sound. (Have been listening to a lot of John Mayer this year and it is hard not to lust for that sound – but I know… I know… the sound is mostly in the hand, and that is true. But a good guitar, a good amp, a healthy dose of placebo effect and some optimistic imagination certainly helps as well.)

ToneStylers are ridiculously expensive for what they are. I have looked to pick up a few older models on sale or used ones. I have a good enough stash now (even for so many projects). And if not, I bought a handful of rotary switches and planning to make my own. (Many on the net described how they built their own ToneStyler clones.)

By the way, a 10 way switch is overkill. Even if you set one at bypass and one at tone all the way up, you would not need more than 4-5 additional sounds. My vote is for the 6 way switch. They do make 6 way switches as well, so that’s cool, though my Strat now has a very old model which was a 16 way switch. It is ridiculous, but it works.

The Productivity Trap

from PhDComics.com

There is a trap that is very easy to fall into in academia. You are constantly under pressure to do more. Deadlines are coming up, the work you have to get through is simply unmanageable. So you get stressed, easily distracted, mentally tired. I keep running for food (which especially sucks on a diet). Have way too much coffee with hopes to jump-start my brain. Even now I am writing this blog post instead of doing what I need to.

When this happens to me, I simply cannot get things done. Trivial tasks, as doing edits to a draft I have already marked up with paper (well, iPad) and pencil. This should be easy.

And the next thing I know, half the day is gone and I made a dozen edits that would normally take me 20 minutes with a fresh mind. Would it not have been better to just take the darn day off and get it all done in 20 minutes the next day with a fresh mind? Yeah, it would have.

Don’t be like me. Take time off! Recognize the trap and do not fall into it. If you do this right, it will make you more productive and you will get more done, on average.