More on AI Literature Reviews in the Social Sciences.

Recently I needed to throw together an overview of the literature for a specific topic. The application was, surprisingly, non-scientific. People who work in the actual world of politics asked me what the literature says about a certain topic. This gave me another opportunity to test AI tools and their capabilities, especially since the topic was not something I was neck-deep in. Simply speaking, I would have had to do a literature discovery from scratch using something like my classic literature review technique if it wasn’t for AI.

If you haven’t read my old piece on AI literature reviews, it mostly still stands. Start there. Not here.

To make a long story short, after trying many things that worked very poorly, (lots of inaccuracies and hallucinations), I finally cracked it and found a fairly good way of doing this. The downside is that it is incredibly work-intensive. Maybe not as work-intensive as doing the classic literature review from scratch, but it really pushed the boundaries of what is possible with AI today and what is not. All in all, I had a lot of fun doing it once I got passed the excruciatingly long and painful preparatory phase.

I tried all the tools in my old piece and then some to try to automate this literature review process. Unfortunately, even scite_ is becoming less and less capable of social science research as they put up more and more guardrails. Its writing is shittier and shittier. Article discovery is, unfortunately, quite poor as (at least without tweaking) it does not find the most relevant literature. It’s still the best thing for discovery but what it writes up is inadequate. (It is good enough for the natural sciences, but not for social science.) I also tried more creative approaches but all of my attempts lead to the AI model just making shit up.

But I recently heard on a podcast, one of the three early April 2024 episodes of The Ezra Klein Show that current models can keep around a book’s worth of information “in mind” while chatting. And since I now have a Claude AI subscription I figured I should put this to the test. I grabbed a whole year of a relevant topical journal. I figured I should try to chat with the model about the articles published in the journal that year. (Yes. Download one at a time. Merge the PDFs and upload.) But I was told it was too much. So I did this for a half year. It worked OK, but it was very slow. Claude 3 told me I could only ask a few questions before it cut me off. Clearly, I hit the limits well before it was useful. And what I really wanted to do is to chat about 6-10 years of issues.

So I tried ripping the PDFs into plain text but I ended up hitting similar limits very quickly. And it didn’t seem to work too well. Looking at what came out of the pdf-to-text auto-generation, I am not shocked the AI didn’t know what to do with that.

Here’s what worked. I just decided to make article discovery mostly “manually.” Fortunately, here the task was either to find everything in the top journals about topic X or glance through everything in the X topical journals that could be used for purpose A. So this was done manually with searches within the journal’s webpages augmented with some specialized Google Scholar work. For the topical journals, I read the titles of entire issues. I also grabbed a few things that the journals recommended along the way in other journals they publish. (In fact, this was done already for months.) To add, I used scite_, I used Preplexity.

After all this, if an article looked like it could be relevant, I took its citation, abstract, and discussion/conclusion and manually pasted it into a text file. I separated articles by #####. This took forever. (In 2024, why did I have to manually copy and paste 60 article abstracts, I have no idea. I guess I do have an idea. Moving on…) But once I had a document with around 60 articles (around 82k words), I was able to upload this to Claude AI and have a good discussion about what is in there. (Pro tip. Never leave a DOI out of a citation if you ever do this. Ask me how I know.)

Yes, every prompt was super slow and I was hitting limits in about 6-8 prompts until it cut me off for a few hours. And this was with the paid Claude AI subscription. So I had to be thoughtful about what I asked. I rolled up my sleeve and used all my prompt engineering (and even more prompt engineering) skills not to waste a single prompt. (If the latter link was helpful to you, sign up for their newsletter. I am not sure I was supposed to give this to you without you signing up. There are many more cool pages where this came from. And if you are at all AI curious, the newsletter is a great quick read every day. ) And sometimes I had to wait until the next prompt… for hours. But it worked. To be sure, first I asked for a bibliography. (That was correct.) And I even got correct quotes when I asked it to go step by step and first identify the main points it will make, then find good quotes in the uploaded txt file to support those claims, and then write up the lit review. Prompts for a full lit review with a few subheadings to guide yielded something much shorter than I wanted (despite size specifications). Going section by section did the trick.

After going through and checking everything (themes the articles were cited for, quotes, cites, bibliography), I can honestly say this worked very well, especially for the narrow purpose. The only issue I had was made-up DOI links when the document uploaded left the DOI link out. The literature discovery was practically manual labor, and the prep work for the uploaded book-length document was mind-numbing, but the results are very good, and overall, it was a fifth of the time to do it this way even including double-triple checking everything.

If it wasn’t so excruciatingly slow, and if I wasn’t running up against limits, it would be amazing to just have a conversation about the pieces with the AI. Have a discussion with the papers or paper authors. In fact, I am now convinced journals should train their models (GPTs?) simply with their own text and allow subscribers to have conversations about the content.

It’s Not Just Trump

You can find the replication information HERE for the Public Opinion Quarterly Article “It’s Not Just Trump” by Levente Littvay, Jennifer McCoy, and Gábor Simonovits. You will find two files here: one replicating the analysis with the Americas Barometer (AB_rep.zip) and one for the original data collection Orig_Data_rep.zip. In the latter, you will find two scripts, one reproducing the analysis in the article and one for the analysis (without independents) presented in the Supplementary Materials. All analyses were done with the most recent version of R, with the most up-to-date packages on January 25, 2024.

Can AI write literature reviews for social science articles? Nope. Not yet.

(Updated October 17, 2023 – still not yet.)

I invested a decent amount of time over the past weeks trying to figure out if there’s a good workflow to follow on AI-assisted article discovery, systematization, and literature review writing for social science academic work.

TL;DR: Nope!

I did this by writing a piece where I was not so knowledgeable about the two dependent variables. I was quite familiar with the 4 independent ones. I needed to review the literature on 8 sets of relationships and the dependent variable in general.

Most LLM tools just make shit up and the verification process to try to figure out what’s real and what’s not and if the real citations say what the AI says they say takes longer (and is substantially more frustrating) than my old-fashioned discovery approach, you can read about here.

There are OK tools for discovery. But they often fail to find the most relevant pieces. But I do recommend you check these out:

  • elicit.org (the new beta is not yet fully functional – but it may be worth buying credits now as it will be more expensive later, I suspect)
  • scite.ai (worth subscribing probably – I did and paid out of pocket, though I didn’t pay full price as I was able to find a coupon.)
  • consensus.app (another one that pays attention to not making stuff up. Good free tier for now – added a month after the others)

Not LLM based, but another useful literature review tool I came across in the process was this: researchrabbitapp.com (And I understand this is free.) Check also: connectedpapers.com

No tool I found was useful in rewriting (messy) notes on relevant articles into a clean literature review draft. It either added stuff it should not have, it got the message, flow, logic, or relationships tested wrong. It failed to recover all the citations from links and notes. I tried ChatGPT, Claude, Llama 2 (of which I run several versions on my own computer), scite, elicit, Bing and Google’s beta search tools. None of them worked very well. Grammarly (with subscription) along with a citation manager and some good old-fashioned writing was quicker and more useful in turning messy notes, and bullet points into good writing. (Well, I wouldn’t say good writing but clear and adequately bland writing for academia.)

This said, I had luck with more organized notes summarization, summarization of PDFs I uploaded which it allows for (including articles I was wondering if I should read fully) using claude.ai. Unfortunately you need to be in the UK or US to use it (or have a VPN that makes it think you are in the UK or the US). Out of all the tools, I liked Claude’s writing the most when I told it to summarize my notes on the literature in the style of a concise social science literature review. 

This note will likely be outdated in 3-6 months (even with this one-month update). The day this changes is the day that LLMs solve the hallucination problem and can, effectively cross-reference links with text written or the day Google Scholar (or one of the others) implements a good LLM. This is not far away at all. I honestly thought we were there anyway, hence the experiment. But we’re not.

Hello Big Name Scholar, Can You Help Me Publish?

I recently had a conversation with a very prominent Hungarian political scientist. (And I mean VERY PROMINENT, someone who has achieved just about everything you can humanly accomplish in Hungarian political science.) We were talking about publishing and how a decade or two ago, he had no clue. He didn’t know what he was doing and given what the state of political science looked like in the country at the time, he had no one to turn to for guidance. There were no mentors. There were no role models. He didn’t know what science looked like, how to engage with it, what one should do to play in the big(ger) leagues. He was mourning the years he wasted doing things that today appear completely meaningless while he could have been publishing in top scientific outlets like he does today.

While the situation in Hungary is improving, and probably in the rest of the region as well, I have encountered places that seem like the Hungarian decades past this friend described. At the same time, I sometimes get truly misguided emails from often advanced colleagues in less developed countries. I assume they email a lot of people, and do not receive too many responses. They are almost certainly also just wasting time. I try to reply to these, but I feel I have little to offer to these colleagues. A recent such contact really made me stop and collect some thoughts on this matter and this blog post is my attempt to be more helpful.

First, if you are a junior scholar, PhD student, or a very freshly minted doctor, it is OK to seek out more senior mentors afar for a mentorship or research visit (though don’t expect those scholars to fund your visit unless you are applying to a call). But if you are more senior, it is most likely that do not need me or my assistance. Maybe you think you do, and yes, there are places where I (or people like me) can be helpful to you, but usually not in ways you ask. You probably do not want to do research with me, you do not want a chapter in my next edited volume, and you definitely do not need my name on your manuscript to help get it published. Let’s look at the flip-side of this as well. People like me have enough stuff to do, we have enough collaborators to deal with, if we are editing something, we will either make an open call for collaborators or we have a very good idea who we want to work with. No cold call will get you closer to your goal. If you think you are someone who can make a substantive contribution to whatever I am doing, you certainly can make a substantive contribution in political science, in general. And that is exactly where you should be starting.

So let’s look at how you can make a substantive contribution. A good contribution is taking some theory, some study out there, almost inevitably done in some weird country (Western, Educated, Industrialized, Rich, and Democratic country) and applying it to your home. A truly amazing contribution, on the other hand, is having the contextual understanding of where your home country is an important case to some theory. Maybe it is an extreme case that should be looked at from the perspective of the theory to see how strong the results can get. Maybe it is the case that will “disprove” the theory or at least question its universality. Maybe it is the critical case. Maybe the empirical result in some weird country is quite straightforward, but when tested in the case you have familiarity, access, insight on, it can truly make or break the theory’s empirical footing. You, coming from a place where political scientists are not (yet) publishing in the top outlets, gives you the competitive advantage to see some new insight and to make a contribution others will not be able to make.

And editors, and I mean editors of top journals, want this perspective in their journals. I have seen editors go out of their way to solicit submissions from people like you. They go out of their way to help get these insights make it to print. They want to lift you up because you have insight, you have a place in political science, and maybe you could become the role model in your region for other researchers who will then deliver even more insight. Even if your attempts are still clumsy, you may have had less training, opportunity, exposure to the publishing game, worse English, etc. they want to see your work in print and they will help you get there. I have talked to editors who set up funds to help with the language of submissions coming from underrepresented places. I have seen editors truly relate to the work of my students from such places differently than from countries or universities they always get submissions from. Know that you have endless goodwill and allies in the community. Your work has to be good enough to be able to get there, but you definitely have allies. Maybe it is not everyone (reviewers are often less helpful than editors but I sincerely believe it should be) but there are a lot of people.

Now, understand that in order to make a contribution along the lines described above, you need to 1. understand the theories and debates in the field, and for this 2. you need to read, and read a lot. You need to read the general journals, the top disciplinary journals, the top topical journals, the top regional journals and read these regularly. You need to make up for the lack of Ivy League education by reading everything those people read and more. And when you find theories that peak your interest, you will need to read the relevant books on those theories as well. This brings me to, what exactly are these journals you should read? How can you identify these journals and the journals you should be publishing in?

I would say the best place to start is Scimago Journal Rankings. They have a list of journals for each field of study and this data (as opposed to Thomson-Reuters) is freely accessible. Look through the list. For reading, focus on Q1 (top quartile) journals. Look at Q2 and maybe Q3 journals too as you seek potential outlets for your work. The top disciplinary journals are well known in each field. Read those for sure. (For political science this is the American Political Science Review, the American Journal of Political Science, The Journal of Politics or if we want to stay closer to Europe the British Journal of Political Science, the European Journal of Political Research, Political Science and Research Methods or the European Political Science Review). Top general journals are less known by social scientists. My go to recommendation for the social sciences is Nature Human Behavior. But general science journals are also worth considering like Science, Nature or the Proceedings of the National Academy of Science (PNAS). When you identify topical or area focused journals, go to their website. Most have at least the titles and abstracts of papers public. Check what they say about themselves. Check the papers that get published there. Is it in line with the kind of work you aim to do? Make a list of journals that are in line and start reading, at least the titles and abstracts. If you can get access, read the whole articles for the most relevant pieces.

Reading these outlets will give you a solid idea on what top scientific articles look like. Be conscious of their structure, how they make arguments, etc. It is also a good idea to have some training in research methods. At least read the books introducing the approaches you want to apply, you want to use so you understand what is expected of you. Seek out all opportunities for methods training.

And then, when you think your work is good enough, go to the submission guidelines described on every journal’s website and start submitting your work. Now, remember, in most fields (law is an exception, I understand) you can only submit to one journal at a time and reviews take many months. So choose wisely. And be productive. Because of how slow this is, you need multiple papers in the writing and publishing process in parallel. Don’t get discouraged by failure. The review process is quite random. A lot of my work is also rejected. Get used to it. Learn from the feedback and submit elsewhere. The first one is the hardest both with submissions and rejections.

And here is where we need to go back to these letters I receive. Please don’t come to me (or people like me) with general requests like the ones I described above. But, it is very true that I can probably help you, but not in the ways people writing such emails usually ask for. So what can you do to maximize your chances of getting this help? Come to me with specifics. I have seen your work on XY and Z and I am now trying to apply what you based article A and B on to [insert country]. I am going about doing this by etc. etc. Then go ahead and ask me what I think about your approach. Ask me if I would be willing to consider helping you in your research endeavors. Explain that you have very few people you can talk to about this topic or even researcher role models to look up to or learn from and I could help in many ways like help you get access to an article or two sometimes (as that is also scarce), answer a question or two when you need someone to help guide you through questions on what your next step or direction should be, ask me if I would take a look at a draft when it is good enough condition (because it helps to get feedback before submission, help with deciding if the manuscript is indeed good enough already or not and what it would take to get it there) or if I’d be willing to listen to you present the research design before starting empirical data collection or willing to watch a conference-style presentation of the paper (as you also don’t have access to the funds to go to conferences to present your work). Because if there is a close topical fit with my work, I will be happy to do this [but please don’t take this as a personal invitation, you really should contact the 3-5 people who are closest to your line of scientific inquiry]. In fact, many, if not most, people I know would be happy to do this. But don’t cast a broad net here. You want to explain (briefly) in your letter how exactly your work fits with mine. (It helps if it fits with my recent work, not stuff I was doing 10-15 years ago that I moved on from.) But I know that with many of these aforementioned emails, I am not the only person getting them. When I respond, often I find that I am not even a close topical fit. Don’t be that person. Put the effort into highlighting why your work is related to mine, I should feel interested in what you are doing and should feel responsible for helping a colleague whose circumstances are not as fortunate as mine. It is time much better spent than producing mass emails that will almost certainly hit spam folders or will simply be ignored because there is nothing in them that communicates why I should respond to you, what your potential contributions are, and there is nothing in there that communicates the kind of competence I look for when I chose my collaborators. If you do the above, you will effectively say, “hey, bigger name scholar, it is worth paying attention to me” and that is what will get you the kind of response you are looking for.

How to Install R and RStudio on a Chromebook (April 2022 edition)

Chromebooks are great computers for students. You get a lot of value for money with a Chromebook. (I am typing this on an incredibly premium feeling 2nd generation Samsung Galaxy Chromebook that was recently on sale for $350 in the US. It has a new i3 chip, 8GB RAM, 128GB SSD, a good screen with touch and USI pen compatibility, it’s light with a flip design, good keys with backlighting and am awesome looking red chassis. Try getting all this in a Windows machine for $350. It is now my daily driver.) But are Chromebooks good for quant-minded people like myself? TL;DL they are! Read to get the details.

Recently someone (a seasoned instructor of R) asked me if I have tips for his student on installing R and RStudio for a Chromebooks. Chromebooks have been my preferred platform for years. The Pandemic unfortunately drove me back to the classic platforms for specialized tasks that are related to the new realities of teaching but I still prefer a Chromebook as my daily driver. I also use R and RStudio regularly. I love the Chromebook simplicity but I also like the flexibility that an on demand Debian Linux installation offers. (Yes, Chromebooks have the ability of running an entire Linux install in the background with all the facilities Linux has to offer.) Of course, I have over 20 years of Linux experience. Not everyone does. And to install R and RStudio, you will have to use Linux and it can be super intimidating at first. So here’s a step by step guide to help you set up and run Linux on your Chromebook, and install R and RStudio. Once these steps are completed, the use of RStudio on a Chromebook is as easy as using any other app on your Chromebook (mostly). I will try to help you both by giving you the steps you need to do and explain what is going on in these steps to help demystify the process. So let’s get started.

(One caveat. This was written for Intel/AMD based Chromebooks. Since, I have tried to do all this on an ARM based Chromebook and concluded that maybe it can be done, but just don’t. Really! Don’t! R’s packaged version installs no problem. The installation of the up to date version is painful, but doable. R-Studio is a nightmare I lived for several days with agonizing breaks in between. And I barely made a dent. I think with enough persistence I could make it work, but it is not worth it, the instructions would be terrifying and would inevitably break with every new version of every dependency and etc. Stick to Intel if you want RStudio and in a pinch, the old version of R works like a charm on ARM.)

Step 1: Tell your Chromebook which Linux version you want to use

At the time of writing I am on ChromeOS Version 100 (Stable Channel). This version of ChromeOS, by default, runs Debian 10 which is nicknamed Buster. But it can run version 11 (Bullseye) as well and you will have better results with the newer version. Before we do anything else, let’s tell the Chromebook that we want Bullseye. My suspicion is that Bullseye will become the default soon. In the future you may not need to do this step at all.

(Note: if you have activated your Linux system on the Chromebook, this won’t work. And upgrading is a nightmare at this point. I’d recommend starting over with Linux by deleting your Linux system. Of course, if you’ve been using that system for multiple things, that’s not a good idea. But then again, if you have been using your Linux, you probably have enough experience to figure out how to install R and RStudio.)

Open Chrome (or a new Chrome tab). Type:

chrome://flags

You should get a list of (mostly still experimental) features you can tweak or turn on and off on your Chromebook. Look for “Debian version for new Crostini containers”. Best if you start typing Debian in the search bar instead of scrolling. Once you found it, set this option to Bullseye (change it from Default).

You may need to restart once this is done.

Step 2: Set up the Linux container.

Once this is done, you need to set up a Linux container on your Chromebook. Open Settings and hit:

Advanced -> Developers -> Linux development environment -> Turn on.

This may take a while. You also may have to set how much space you allow for Linux. Best to use the default and this can be changed later but I’d say 5GB should be more than enough. (Use 2GB if you are strapped for space.) Once this is done, you should have an icon looking something like this in your menu.

This icon will open a Linux Terminal, allowing you to work inside Linux. (Not sure if it is automatically pinned into the shelf on the  bottom. If not, consider pinning it.) We’ll need to hit this next.

Step 3: Open Linux, and do the mandatory regular maintenance. 

When you open this, you will get a black box with a blinking cursor that you can give commands to. People experience with Linux are probably quite used to working in this so-called Terminal. But if you are not, this can be intimidating. Don’t worry. We don’t have to use this often.

You want to basically update your Linux. Make sure everything is up to date. You do this by typing:

sudo apt-get update && sudo apt-get upgrade -y

(If you care, sudo do the following as a superuser who can do anything, normal users can’t install things – sometimes you may have to type your password when you give this command) apt-get is what manages everything installed in Debian Linux and update means check what packages are on the server.  && means I am giving another command now, which is the same command with upgrade meaning upgrade all packages that are installed and newer versions are available on the server. -y is not that important. It basically says, don’t ask questions during this process, rather say yes [or whatever the default response is] to everything.

Step 4: Install R

Bullseye has R 4.0.4 built in. [This is what works well on ARM as well.] Buster has an even older version. 3.x.x I suspect. For an intro to stats class it may be enough. (To install this outdated version, type: sudo apt-get install r-base r-base-dev libnss3 and move to the next step. And to explain what this command does, you are already familiar with sudo and apt-get, install installs stuff and r-base is the name of R in Debian and r-base-dev is an extension that allows you to install packages, finally libnss3 is needed for RStudio but I couldn’t figure out why). But you probably want the most up to date version of R running on your computer [which at the most recent update of this post is 4.2], not whatever was originally included in this version of Debian. So we need to jump a few more hoops to make sure it is not the default version of R that is installed, but the new version.

Remember that we talked about updating what packages are available on a server? We can add a new server here pointing to the most up to date version of R. R actually maintains a server with which you can keep R in Debian up to date. To make Debian use this server you need to do two things. Add it to the list of servers Debian checks when you run apt-get update, and authenticate the server ensuring nobody can hijack it and install malicious stuff from there. (Linux is pretty secure from this perspective.) To do these we need to install a few other things we will use in the process.

(Note: when you install stuff, sometimes there are so called dependencies that also need to be installed before the stuff you specified can be installed. So don’t worry about it if Debian is telling you more stuff will be installed than what you asked for. This is normal.)

sudo apt-get install dirmngr gnupg apt-utils apt-transport-https ca-certificates software-properties-common

With these installed, now we can authenticate the server we are about to add (that big ugly string is the key for the authentication – just go with it).

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key '95C0FAF38DB3CCAD0C080A7BDC78B2DDEABC47B7'

Then we add the R server to the list of servers Debian should update from

sudo add-apt-repository 'deb http://cloud.r-project.org/bin/linux/debian bullseye-cran40/'

Then we update the list of what’s available on the servers. (We’ve done this before. But now it will include the new server we just added.)

sudo apt-get update && sudo apt-get upgrade -y

And we install R the same way we would have installed the default (older) version. If you did that before, the previous step will update the already installed version of R to the newer version. So no worries there, you can skip this. Or don’t. It will just tell you that r-base and r-base-dev are already installed.

sudo apt-get install r-base r-base-dev

Finally you need to install libnss3. (Don’t ask why or why it doesn’t install it as a dependency. Definitely don’t ask what it is. No clue. But I had trouble with RStudio without installing it. Maybe R would have issues too. I didn’t check. But multiple sources suggested this needs to be installed in addition to r-base and r-base-dev.)

sudo apt-get install libnss3

Step 5: So R is installed. Now you need RStudio.

RStudio must be manually downloaded from the RStudio website and installed also manually. So go to the RStudio website and download the free RStudio Desktop for Debian 11. (Same version for Ubuntu. You may see it listed as Ubuntu/Debian.) Link at the time of writing is here: https://www.rstudio.com/products/rstudio/download/#download

What you downloaded is a file ending with .deb (this stands for Debian package). Put this in the home directory of your Linux file system in the files app.

When you want to install such a deb file, and not something on a Debian server, you have to tell apt-get where the file is. The ./ in front of the file name means in the folder I am working in (as opposed to the server where it won’t find it). Make sure the file name matches what you downloaded. This is the up to date version at the time of writing, but RStudio updates regularly so chances are you will be grabbing a newer version with a newer date and version number.

sudo apt-get install ./rstudio-2022.02.1-461-amd64.deb

Et voilà. RStudio is installed. It should be in the list of Chromebook icons called Linux apps. Run it from there. If you are going to use RStudio regularly, I would pin it to the shelf on the bottom. And again, you will not have to (often) do anything in the Linux terminal. There are minor exceptions I will list below. Once you verified RStudio works, you may also want to delete the deb package you installed it from before you close the terminal. Keeping that is a waste of space. 

sudo rm rstudio-2022.02.1-461-amd64.deb

Additional Info

As a general good practice, I always make a folder in my Linux partition I call “data”. For all R projects I make sub-folders in this “data” folder. In the R code, if you want to refer to a folder in R with the setwd command you do it by

 setwd(“~/data/project1”) 

assuming the sub-folder for your first project is “project1”. ~ means in my (Linux) home directory and the rest is pretty self explanatory (I hope). This is going to be an important folder. Remember to back it up regularly. I occasionally dragged this over to my Google Drive to make a copy. (Though, now I am working on setting up something more automatic that syncs this folder across all my devices and the cloud. UPDATE: mega.nz ended up working great for this. I had less luck with pCloud. I should be recommending GitHub here, but if you just want a quick and dirty solution, Mega works.)

Now I said above that you will not have to do much in the Linux Terminal. There are three exceptions. Once in a while it is a good idea to update your Debian packages. Open the terminal and:

sudo apt-get update && sudo apt-get upgrade -y

That’s it. This will update to the newest version of 4.x.x R and update all other packages installed to the newest supplied by Debian Bullseye. (Once R moves to 5.0, the list of servers will need to be updated and R, along with all of its packages, reinstalled. But that’s gonna be a while.)

Sometimes you may want to update your version of RStudio. Repeat Step 5 once a newer version of RStudio is available. You can check the current version of your RStudio under Help -> About RStudio. Version mirrors the file name.

Finally, when installing packages in R / RStudio, sometimes R will tell you that it cannot install something because it requires tools not available. See example below. 

See how it said “Configuration failed because libcurl was not found.” That is your cue that something needs to be installed still. But (despite the intimidating mess around it) R is really clearly telling you what you need to do. On Debian or Ubuntu you need to install the libcurl4-openssl-dev package. (These are always packages available within Debian.) And, if you paid attention above, you know how to do this already. For the example in the picture, open the terminal and:

sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install libcurl4-openssl-dev

And that’s it. You just need to not panic and find what went wrong with the given R-package installation. It is almost inevitably a missing package that you need to install and it tells you what it is. If this was not the issue, find whatever clue R gives you and Google is your friend. If that doesn’t help, turn to the forums online. They will help.

Conclusions

In sum, Chromebooks are just fine to run R. And Linux is generally a good platform for most quant work. Chromebooks give you a full Linux install which is an extremely flexible tool for data science. Many people opt to use just Linux for their data science needs. With a Chromebook, all that power is available at your fingertips on demand while you have a pretty simple to use computer where the tech and the platform does not get in the way of your day to day computing tasks.

I honestly don’t use the Linux for much,  but I could. They are working on it but Audio is still an issue with Linux on Chromebooks but soon I expect I’ll be using Audacity for audio editing and maybe even a video editor. GIMP is a good Photoshop replacement that you might prefer over web apps. What I do have installed in addition to R is LibreOffice which is helpful. I usually use Google Docs but sometimes having a second platform helps. I also have GNU PSPP (an SPSS clone) installed though I barely ever open it. I keep an extra browser and text editor, I have an FTP client (FileZilla) and Signal for desktop installed.  These are some additional tools you can use with Linux.

Sources

https://beebom.com/how-update-debian-bullseye-chromebook/ 

https://www.how2shout.com/linux/install-the-latest-r-programming-language-version-on-debian-11-bullseye/ 

https://www.linuxmadesimple.info/2021/03/how-to-install-rstudio-on-chromebook.html 

https://community.rstudio.com/t/error-in-install-packages-ubuntu-20-04-lts/67443 

https://community.rstudio.com/t/installation-error-cannot-find-libsmime3-so/30646 

And thanks to Jeff Lessem for the tip on how to make this even more simple.

Hybrid Meetings

As the post-pandemic world moves into a new stage, a new genre of meetings is emerging, hybrid meetings where some participants are in a room with mics, cameras and technology, while the rest of the participants are joining from afar. When meetings moved on Zoom, we quickly learned to mute our microphones when not talking, to use calendar invites with meeting links embedded, we learned to raise hands, use emoji symbols, instead of speaking over each other and to use the chat when appropriate. We learned how to look, act, and be professional in such meetings. The Internet was quickly flooded with advice and best practices so resources were plenty. Unfortunately the movement towards hybrid meetings has not mirrored the quick learning with ample information available upon a quick search. This post is a first attempt at remedying this.

I partook only in three hybrid meetings since meetings could take place where there were a reasonably large contingent both in the room and afar connecting through technology (Zoom, in this case). And these meetings sucked. They mostly sucked because people in the room paid little regard to the people who were participating through the technology. I understand why. They were caught up with the local social interaction. No judgement in this message. But the situations clearly showed that, to make such meetings viable, we need to consciously consider all meeting participants and conduct meetings accordingly. Here are a sampling of the mistakes and recommendations for overcoming them from these few meetings. This is far from an exhaustive list, but hopefully it will serve as a conversation starter. (Feel free to send me more points to add.)

    1. Please begin the meeting on time. If for some reason the meeting cannot begin on time, please inform the people joining from afar so they don’t just sit there disoriented not knowing what is going on.
    2. Mute the main room microphone until the meeting is officially ready to begin. I have listened to over an hour of 15+ people engaging in independent chit-chats around the microphone. (Yes. One of the meetings started that late. And we could only guess what the hell was going on as we sat there for 10-20-30-40 minutes listening to disorienting chatter.) The noise of multiple conversations through a microphone and a headset is extremely disorienting, stressful and unproductive. I could turn my volume down but then I don’t hear if the meeting actually starts or I am spoken to, so I was worried to fix the problem on my end. Had I known what was going on, what delay is expected, I would have done so, but I didn’t (see point 1).
    3. Organize the room so that any guests of honor at the meeting, or anyone whose role is to officiate or is expected to say a lot (not to be confused with speak a lot), do not end up with their back to the camera. I spent two out of the three meetings looking at the backs of the most important guests in the meeting.
    4. Watch for interventions by those afar. (One of the things that helps a lot with this is point fixing #3.) The raise hand function should work well even in hybrid meetings as long as someone is watching for the raised hands. At the same time (to also offer advice to those being afar as well) the use of the chat is useless. Nobody in the room will see it. So, just don’t.
    5. Try to actively bring people into the meeting who are only there from afar. Ask them explicitly. Introduce them first before the people in the room are introduced. In hybrid meetings, there is a need for someone to be a firm master of ceremonies. Say what will happen now. Say it clearly, slowly, articulately, into the microphone as you are looking at the camera. Watch for feedback on the screen to see if it was understood. It does not help if the MC says “we start”, in the middle of the chatter chaos, because that message will simply get lost. Call on the people on the screen one by one for introductions. In one of these meetings I was at, the introductions went to the room first and the people afar really wondered if anyone knew we were there. It was over an hour before any interaction from the screen was even possible. In another such meeting those afar went first. But I did not even catch that we were starting, or doing introductions. I had no idea if I was supposed to just say my name or more. It was probably said, but with 5-6 other conversations going, nobody who was not there heard it.
    6. When using a shared screen in the room, don’t assume people see what you are pointing at. Any visual aids held by people in the room (like holding up a book) will be lost through the camera as in hybrid meetings people are tiny on the screen anyways. In one of my meetings there was a shared document and somebody was clearly standing in front of a screen pointing at things, but nobody who was joining from afar, saw what.

These are just some of the basics of how such meetings could be made enjoyable. Of course, advice #1 is: be there. This was not an option (or in one of the three meetings, a priority) for me unfortunately. In a different world, I would have just missed that one. Alternatively, organize online only meetings. It works better. But I understand why all of us are eager to get back to a normal life where Zoom is minimized. Finally, consider just not being there if you have to Zoom in. (This is what I should have done with the one meeting for sure.)

Unfortunately, such meetings will become the norm. I think they could be done well. But we need to learn how to do such meetings well and do it quickly to maintain our sanity.

How to Do (Non-emergency) Online Education in a Few Videos

Source: Ken Schultz Tweet

This post is designed to be a non-comprehensive collection of useful information and tools to produce quality online education. As one of the most international Universities in the world, CEU is looking to duplicate a large proportion of their courses online for incoming students who may or may not be able to come to our brand new Vienna campus and take part in our traditional courses. CEU is quite different from most universities. We only do social sciences and humanities and we only have grad students. (Starting this Fall we will also have undergrads, but the vast majority of our training remains grad training.) Needless to say, most of us were also not prepared for the disruption in how we do things earlier this year. I personally would not have known anything about this either but while many of my colleagues were reverting their courses to emergency measures, worrying about what they need to do tomorrow so that their classes do not fall apart, I was not teaching. Still, my circumstances required me to figure out the online education thing fast. (At least I had the time to do it.) 

I am putting together the most useful videos I found to help people visualize, conceptualize and deal with the task at hand. I am also sharing experiences in the text. Hope it will be helpful.

A Bit of Background

(Feel free to skip to the next section if you just wish to quickly get to the resources.) 

Along with two colleagues, Derek Beach from Aarhus University and Benoît Rihoux from the Catholic University Louvain, we have been responsible for the academic content of the European Consortium for Political Research (ECPR) Summer and Winter Schools of Methods and Techniques, two events that run over 100 courses and service close to 1000 people every year for graduate level social science methods education. By March it has become blatantly clear that ECPR will not be able to run the Summer School, hosted by CEU since 2016, in its traditional in-person format. The three of us, with the help of one of our instructors, Cai Wilkinson from Deakin University in Melbourne – famous for its online Cloud Campus, spent what feels like a full time job rebuilding our Summer School offer, recruiting instructors who were either experienced with or excited about investing in high quality online graduate social science methods education. We have been playing with the idea of developing an online offer to complement our in-person event maybe for 2022/2023. (We even had a meeting planned for late spring in Florence, Italy to brainstorm it out. Needless to say, no in-person meeting took place in Italy but the timeline accelerated fast.) We quickly had to become experts in the pedagogical model and get creative in sorting through what would and what would not work for graduate social scientists. We personally ran two pilot workshops (one on Process Tracing and one on Multilevel Regression Modeling) to test out ideas, infrastructure and to gain hands on experience with the pedagogical model. Our instructors are looking to us for guidance and we are preparing the materials to support them. Our new event, on opening day registered people for ¾ of the slots available and landed 40+ people on the waitlist for their preferred course. Now we are working on expanding our offer to accommodate everyone else looking to participate. At the same time CEU made the decision to duplicate its offer between online and on-site for the Fall. For these two communities, I am hoping the following materials will be a useful curation of the hours and hours of videos I watched trying to sort out what is useful and what is not.

First, while the emergency teaching happened synchronously as all of our students were at either our Budapest or Vienna campuses, this model will not be the most appropriate for the Fall when most of our students will be scattered around the globe. We need a model that overcomes tech and connectivity issues people may have. The best way to do this is to rely heavily on asynchronous components of teaching, augmented by synchronous interactive components. It is important to ensure that students feel connected to both the instructors and to each other, to feel like they still are part of a learning community and receive a high quality educational product. Tools that best achieve this are tools we could and should have used in our regular classrooms for a while already, we just did not know about them. Many of these tools will be useful to enhance also our regular teaching. 

(There is a lot of research around pedagogy. Here I will predominantly focus on the basics, the technology, the hands on how-to and the tips and tricks to improve production value of the course. CEU people, please check out the Center for Teaching and Learning materials to get a more comprehensive picture on pedagogy. They have a course on Moodle with great resources.) 

Moodle

Most of us are familiar with the tool (or some other kind of Learning Management System like Canvas, Blackboard and etc.). It is important to have a centralized landing platform for students to access the courses and these LMS systems serve this purpose well. For us at CEU, this has been Moodle (or as some of you would know it, e-learning and later ceulearning, both built on the Moodle platform). This will continue to be the foundations of online classes. The functionality of Moodle is quite endless and I know for a fact that the CEU Moodle team is hard at work integrating the below mentioned tools (and more) for smooth interoperability, along with also upgrading the platform visuals for a more modern look. 

Asynchronous Teaching (The Lecture)

The idea is to separate the teaching of the material into lectures that are unidirectional knowledge transfers which can easily be done asynchronously and other sections (such as social learning, group work, in class discussions, Q&A and etc. – more on this later). Even in regular teaching times, the, so called, flipped classroom model is a useful tool. There, the pre-recorded content is short and lays the foundation of the in-class discussion or activities. With a fully online course, a pre-recorded lecture will be longer, though not as long as an in class lecture in a small group where discussions interrupt the material the instructor plans to deliver.

A few tips on lectures. Again, (1) If the material can be divided into smaller parts, divide it. It is best to have 10-15 minute bits. If the material cannot be divided naturally, that is fine as well, but use division when you can. (2) If your class is in the 15-60 people range, you probably still have some interaction during the lecture. Your recorded lectures will (and should) be shorter when covering the same material. (Estimate roughly half.) In fact, on a video you may want to go a bit faster than in a regular in-person lecture not to bore people. (Remember, people can pause the video. (3) Try to speak in an interesting way. Try not to be monotonic. And, if it is possible, provide closed captioning. There are a few tools out there to do this. (Not sure if Panopto will do it. YouTube does it automatically. And you can find some additional tools if you look.) 

Panopto

Here are some materials that will help you visualize what a good online lecture could look like without professional production facilities. Many (paid) online courses or monetized YouTube tutorials rely heavily on a high production value, they are recorded in a studio, produced professionally, but this is not something most of us will not have the ability to do. It is also not necessary to offer quality. Remember, quality comes from the content. 

CEU has a Panopto license which will be at the core of strategy to help people move their lectures online and it is truly a great tool for this. Here’s an example of a Panopto lecture (recorded probably without much special equipment like a laptop camera and laptop mic – you can of course up your game with a better camera, microphone and lights). This example uses PowerPoint Slides but a board is also available which could be recorded on a Smart Board in the building or an Stylus supporting tablet (sorry, I don’t have such a video):

https://demo.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=1e9a9689-9882-4625-9471-444b4c9bb54e

Panopto can even be used more creatively. Not so much applicable for the social sciences, but look at this biology class utilizing a microscope with a built in camera.

https://demo.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=c1cfa29b-2bc7-4739-8e08-46143d9d1e55

Panopto can augment videos with quizzes pointing to the most important materials guiding the students what should be at the center of a potential live discussion, for example on Zoom. Here’s how:

https://www.youtube.com/watch?v=TAPrM8jHTtM

Panopto can also be used to record in-class lectures. Less than ideal for the asynchronous format, but in a pinch, it could work. Separation between (the recorded) lecture elements of the courses and the interactive component of such recorded classes is key (like one would do in a large auditorium). Classes where the flow is constantly interrupted or that are seemingly unstructured discussions are not ideal for recording and yield a poor experience for the students watching the recording. So some adjustment may be needed when one decides to record an in class lecture. But it is the same adjustment one would do to talk to a large crowd in a large auditorium. Maybe we are not used to this at CEU where class sizes are kept relatively small, but we certainly can easily make this adjustment.

If Not Panopto

If someone wants to move out of Panopto and use some basic video editing tools, it is entirely possible to set up videos of different styles. This one is a particular favorite of mine (a video ironically reviewing Massive Open Online Courses). 

https://www.youtube.com/watch?v=0fp60iHV7Rk

I teach quite a bit of statistics. For this type of material, I am a fan of the Khan Academy format. Here’s a course with material that could have easily been in one of my classes (if it wasn’t for the fact that my handwriting is so horrible).

https://www.youtube.com/watch?v=DaBq0naj0YY

And you can always just record yourself giving a talk and place it into the corner of a PowerPoint. To best make this work, use a 16:9 aspect ratio for the slides (because this is the size of videos) and leave an area for your image to be inserted. Pro tip: if you make a background with a small dark box the size of the recording you use, you will always know what area you need to leave blank on the slide.The tip also works for handwriting on smart-boards and tablets. Just tape off a corner and then you will never write over that area..

These types of videos require more video editing. For this I would recommend some free tools: iMovie (on a Mac) and I have been recommended Movavi for Windows (but will have to report back on how it worked once I tried it – yes, I intend to test it soon). Sometimes you just need audio editing options. For this, I recommend the free, platform independent, and incredibly flexible Audacity.

At the end of this post, I will also include additional tips for improving the production value of pre-recorded materials, how to think about potential equipment use and even some DIY tips on how to make equipment on the cheap. But first I would like to get to additional components central to the learning experience.

Other Tools To Augment The Asynchronous Experience (H5P and Perusall)

Panopto offers you embedded quizzes, but these other types of videos can also be augmented by an even larger array of activities. Look into H5P for this, which CEU’s Moodle fully supports. Here’s a good introductory video.

https://www.youtube.com/watch?v=2HtxLeXGU48

I can easily say that the coolest tool I came across in my search was Perusall. With it, you can turn the reading experience into a social experience. When access to the instructor is limited by distance, it is advisable to annotate assigned readings, ask questions, make comments to the students. It enhances the experience of the students feeling like the instructor is specifically talking to them commenting on the readings. (Also, if you ask some easy questions, it may help everyone feel included. Not just the most advanced students. So try that.) While this can be done easily in a PDF, Perusall takes this to a whole new level where students can also, socially, comment and discuss the reading materials. Here’s an introduction of the platform. (Sidenote: one of the developers of this platform is Gary King, a political scientist and methodologist at Harvard.)

https://www.youtube.com/watch?v=bxEfWdfxj28

While clearly developed for undergrads, I have had luck using Perusall with grad students. Their business model is to sell textbooks to students, but they also allow the uploading of PDFs. If you assign journal articles or book chapters, this also works great (though stay away from poor quality scans and we will need to sort the copyright issues, but I understand the CEU library team is already on this). I did a webinar with Harvard Physics Professor Eric Mazur who is one of the brains behind the platform. He gave a few recommendations. (1) Don’t comment on what students say. You do not want to create an environment where students ask questions and the instructor offers the answers. (Or worse, an expectation that you will be present 24 hours a day to respond.) Let the students discuss. (I suppose it is OK to upvote or star when someone gives a thoughtful response. And in my workshop I offered responses but only after the lecture where we covered the material at hand.) (2) Turn on the Machine Learning algorithm for grading. The platform can actually grade students automatically (if the group is large enough, if I recall, 15 is the lower limit). I suspect this may not work so well in graduate school. I don’t suppose the platform can distinguish between high level of engagement and highly articulate nonsense and highly articulate thoughtful comments, but for the rest of the spectrum (inadequate engagement) it should be good. It is probably good up to the 80th percentile of the grading scale and the rest needs to be bumped manually for especially thoughtful comments (which makes the work easier for sure). 

Synchronous Components

And there are components of the course where synchronous communication will need to take place. The biggest challenge here is time zones. Departments will have a hell of a time scheduling synchronous components for multiple time zones. But it will have to be done and some synchronous class components will have to be done twice.

Zoom (please, let’s stop using Teams)

I realize Zoom has been controversial. This is mainly because of the publicity the company received for Zoom bombing (people crashing meetings immediately broadcasting pron or racist content). The reality of this situation is that it could have happened on any other platform where open links were broadly distributed. Zoom was just the most used once the pandemic hit because it was the best, it ate the lowest amount of bandwidth, and it had the least amount of lag issues. Zoom is very flexible allowing for an array of interactions between instructor and students during a traditional lecture (though this format should be avoided) including yes, no, go faster, go slower, thumbs up, clap, raise hand. Features such as break out groups offer powerful tools in teaching. (Yes, we tested it in our pilots with excellent results. It is even useful for social activities where with 20 people everyone is quiet but in groups of 6 the conversation is vibrant.) If one knows how to use Zoom’s settings, the security issues go away. Other issues that point beyond Zoom bombing have been addressed in Zoom 5.0, so make sure you have an up to date version installed.

Zoom is useful for live Q&As, class discussions, synchronous group activities and all forms of interactions you may want to do live with the students. It can be used for lectures as well though if one is going to lecture, it is best to just pre-record it.

Here are the video resources I recommend for getting up to speed with Zoom.

Managing Participants: https://www.youtube.com/watch?v=ozJS9bvdVp8

Break Out Rooms: https://www.youtube.com/watch?v=jbPpdyn16sY

Sharing Your Screen: https://www.youtube.com/watch?v=YA6SGQlVmcA

How everyone can share their screens (useful for coding classes): https://www.youtube.com/watch?v=pt-tcvaQ9I4&feature=youtu.be

Using a tablet as a white board in Zoom: https://www.youtube.com/watch?v=SCr010CMRsg

And there are additional tips and tricks we should keep in mind. First set the tone for your workshop. You will have to walk a fine line between not being intrusive to people’s lives but insisting on them being present. Courses work best when everyone has good bandwidth, everyone is engaged and everyone has their camera on. Unlike in Teams, you can put a lot of faces on one monitor in Zoom. If there is some reason they need to turn their video off either temporarily or for a whole session, it is fine but try to avoid students falling into the habit of participating with their screens off (if bandwidth allows). How you set the tone about this early on will matter. Etiquette also calls for you looking at the camera when they are talking. (This is extremely hard when your screen shows, say slides or shared screen, and their faces are on another screen next to you.) Everyone should mute their mics when not speaking. (And Zoom allows the host to mute them as well if needed. Sometimes people forget their mic on and start to have a conversation with someone who walked into the room.) Obviously people should only use the chat for useful stuff. As instructors, it is extremely hard to monitor the chat while teaching. But it is still useful for the students for interaction. Don’t host alone. Hopefully you will have a teaching assistant. But if not, plan roles for the students. Ask them to help each other with technology issues. (This was especially important when teaching the software component of a class.)

Slack

Slack is an excellent software to build a learning community where people can interact in various contexts. It would give a platform for people to discuss the class. Instructors can chip in. (I would schedule set times for the instructors to answer questions. But you can also easily follow the fluid class discussion and put a like or heart next to good answers people give to each other, offering some insight if the discussion is heading the right or wrong direction.

Here’s an example video of how one person uses Slack for a class. But I could imagine a whole department setting up a common account with all the courses, with rooms for tracks, specific courses, specific instructors where questions can be posed to them.

https://www.youtube.com/watch?v=8K_N0YCBktU

Tech Tools for Teaching Coding

For more technical courses which require statistical analysis or coding (which may be relevant for Econ, Network Science, Cognitive Science, Public Policy or even some Political Science Classes) there are multiple tools available. Building your own server based infrastructure can certainly be very helpful. I have done this before with R-Studio Server when I needed to teach in a lab where I did not know what will be installed or even if users will have permissions to install, say R packages. The issue is certainly similar on whatever computers people use at home. Amazon AWS infrastructure can be very useful in these instances (if you don’t have your own server). I am in the middle of exploring more such options. On my radar are rstudio.cloud, codeshare.io and I have personally used Google Colaboratory.

Google Colaboratory (or Colab) was designed for the teaching of machine learning applications in Python. Here’s how it works. You upload (or create a new) Jupyter notebook in Google Drive which you can open with Colab. Inside you can use a very simple markup language for communicating information with students in an organized way, you can embed code which you can run on the spot on their free server. It is not very fast but it offers RAM, CPU and GPU. And there are other similar platforms out there as well (but I did not go down this rabbit hole… yet). See here:

https://www.r-craft.org/r-news/six-easy-ways-to-run-your-jupyter-notebook-in-the-cloud/ 

and here:

https://analyticsindiamag.com/5-alternatives-to-google-colab-for-data-scientists/

And now, Colab also works with R. It is a bit annoying how packages you may want to use (and are not installed by default) you have to compile from source (just like in R on any Linux machine). But beyond that it was pretty easy to use. Here’s a screenshot which may help you imagine how it looks and feels a little better. Once the notebook is created it can be shared both online and offline. And the students working in their notebook in Google Drive can easily share their work with the instructor/TA “just like how one can share a Google Docs document and even work in it simultaneously.)

One big downside of this platform is that it requires a Google account. But I have no problem asking someone to set one up just for the class, even if they are opposed to having a Google account (or do not wish to use their personal account for work/school purposes). It may cause some friction.

There is one big caveat to these technical components. We need to be very serious about moving to accessible platforms for everyone. This means a painful (for many) move away from Stata. Students will not have licenses. We cannot rely on computer labs. (Those things are a thing of the past.) I know Stata (for me Mplus) are sometimes the best tools for many jobs, but in our teaching, it is crucial to move to platforms that are open and accessible to everyone.

Production Value of Videos

Finally, I have also compiled a few videos with advice on how to improve the production value of video recordings.

How to build a video studio at home (with great discussions on shooting, background, placement, lighting – no need to take all the advice but it is a good list of what to think about): https://www.youtube.com/watch?v=TAw-xGSd83I

Cheap gadgets that can help (not necessarily these specific ones, but ones like these): https://www.youtube.com/watch?v=ruDJgtcZB-E

If you want an alternative set up for a white board (that is not just your tablet) this is a good one (especially the second set up presented which only uses a mobile phone camera). The video is intended for people who want to record drawings, but I could easily see it as a whit board alternative. https://www.youtube.com/watch?v=uzv8bxqvXy8

How to build a cheap studio light: https://www.youtube.com/watch?v=IFON04_L3zk

How to make a cheap green screen: https://www.youtube.com/watch?v=4wySe_NqfEU

If you want to know more about cinematic lighting, and how people use it for youtube (or teaching) videos, this is a good introduction: https://www.youtube.com/watch?v=G6W5wbPqzPw 

I use a DIY studio key light built from a metal cookie tin box and a ring light from the cheap gadgets video as a fill (in a basement room with practically no natural light). Nothing else for now, but working on it.

Thoughts on CEU: CEU should close the Budapest campus

I am going to preface this by saying, I am putting my personal opinions in writing here. No more, no less. I don’t speak for anyone else. My words carry no more weight than anyone else’s who is part of the CEU community. I know this post will piss a lot of good friends and colleagues off. I hope you can forgive me if you are one of these people.

I started writing this series with two interrelated goals. I want to make sure the long term financial health and stability is at the forefront of more people’s mind. I also want to find a way to achieve this financial health without hurting our character and academic prestige in our new situation (which, I am sorry to say, is in Vienna). The first post in my series focused on a possible avenue to achieve this. It discusses a recent innovation in education that has the potential to generate more revenue than a tuition scheme and do so without sacrificing who we are. (If you haven’t read that, please stop reading this and read that first.) The second was a more personal (and, despite having read by more people than the first, probably less of a general interest) post on the consequences of what a good friend called the “academic vandalism of Fidesz and Viktor Orban” also highlighting some of my fears of how the move to Vienna could go terribly wrong for CEU.

There is an ongoing conversation about CEU’s presence in Hungary after the Vienna move. There is one proposal on the table arguing for maximal presence in Hungary. There is a working group put together right now to develop the plan for activities for the Budapest campus. As far as I can tell, there is not a voice on the other side, so for the duration of this blog post, please allow me to be that voice.

There is very little conversation on what the costs of maintaining operations in Budapest are. The proposal on the table does engage in some wishful thinking on where we could maybe get resources for this, but it is as realistic as the magical thinking about the revenue generating abilities we imagined for the alreadz defunct CEU Business School, or for the once overgrown and now deflated School of Public Policy, or, dare I say, our imaginary new tuition generating Viennese programs with which we can grow out of our financial deficits. (The rector, before the crisis, took office with a plan to shrink. That sounded prudent. Whatever happened to that plan?) We don’t have a good track record here so I fear for our future. But even if we accept the arguments made for revenue generation of a Budapest presence, it is not like most of those resources could not be generated in Vienna or put to better use on other projects. Nobody is considering the opportunity costs.

First, every day we remain in Hungary, we validate Viktor Orban’s arguments. He has been going around saying: “What do you mean we kicked CEU out of the country? They are there. Right there. Go down to Nador Street. The sign is on the building. People are going in and coming out. CEU never left Hungary. What CEU said is just a trick to smear the Hungarian government. They are not going anywhere.” And every day we stay in Budapest, Viktor Orban is right, we validate his rhetoric. Our complete existence is made impossible in Hungary, but we remain stubborn, stay and help Viktor Orban make his point. The University is now desperately searching for things to do on the Budapest campus without considering the possibility that maybe we shouldn’t do anything. Maybe we should leave. Seemingly, we are committed to taking a stand saying we are not going anywhere, but are we really taking a stand here? Maybe leaving would be the actual stand.

I understand that people have strong connections to Hungary. I do too. (If you doubt it, then read my previous post on what the Vienna move means to me.) But anyone who believes that a move to Vienna is evitable, if they can avoid commuting, they are really fooling themselves. A better question even, would they be so committed to Hungary if they had to commute back to the Budapest campus from Vienna?

I love our new building. It feels like a second home to me. I do not even go to my office (or, for people who really know me and know I never worked in my office, I also don’t go to the nearby cafes) anymore. I just work in the N13 public spaces because they feel like such a great work environment. I love our classrooms. I even loved teaching 60+ people stats in our 76 seat auditorium, because the room made it feel like a 25 person class. (That was some serious efficiency gain for teaching credits. I’ll never do it again without excessive TA support.) But, I believe, we need to go and make an equally great place, great environment for CEU in Vienna.

The second argument for leaving is financial. We simply cannot afford to maintain a presence on a second full size campus (that we are desperately struggling to fill up with life with things that will also cost money most probably). As far as the finances of the University are concerned, the people who are putting plans together for a tri-campus life with an equal Budapest and Vienna presence often cite the recent generous donation by the founder. If anyone thinks that generous donation is good for anything but putting us back on life support, really needs to check their numeric literacy. (I welcome them in my intro to stats class next year. We start with a basic algebra refresher.) Let’s just say, they need to have serious conversations with the Provost who is responsible for CEU finances. In a phone call we had recently, he was able to put our financial situation in quite real terms. I mean, we will survive it, the situation is not that dire, but… well… see the first post in this series about our long-term financial health. (And maybe also the second on my associated fears.)

There are two costs associated with the maintenance of operations in Budapest. Everyone talks about one of these, the cost of the actual operations. But there is another big ticket item that economists dub the opportunity cost (or the cost of not spending these resources somewhere else). To put it bluntly, I strongly believe that CEU would be better served putting the value of the real-estate portfolio back into the endowment (and to do this while real-estate prices are so high in Budapest). If the money is in the endowment, that increases our annual yield, it gives us more money every year to operate. Maybe the pressure to charge tuition would not be so large then. Maybe we could remain that place that is open to everyone if you are smart, hard working and ambitious (even if you do not have access to loans or do not come from money).

I understand CEU wants to maintain some of their non-academic staff in Budapest as a cost cutting measure. (Staff in Vienna will be expensive.) I am all for it. Let’s rent office space somewhere for them. We don’t need to have these offices in some of the most expensive real-estate in Budapest.

What makes this proposition quite painful is that such a wholesale departure, commitment to give up the downtown campus, means that our departure is final. But what were we expecting? That we can pack up Vienna and move back? Or that we miraculously could afford to operate two equal campuses in the future? (Check how that worked last time.) I understand that our new (social science PhD) mayor is now suddenly very welcoming but how does that help us when Fidesz, as they seemingly take a step, or two, back from their usual institutional annihilation march, don’t you all find it curious that the already negotiated treaty, the only thing needed to ensure CEU’s existence as an American university in Hungary is, at least to date, is still not signed by Viktor Orban?

I love our new buildings as much as anyone. But in my decade and a half at CEU, we already toyed with the idea, once, that we get out of the real-estate and move to another location. If we can ever afford to come back to Budapest (and the situation is such that we are welcomed back with no risk of a single election swiftly changing that), we can find a new home for CEU in Budapest. It is not like this possibility was not on the table before (although, I remember people didn’t like it too much then either, but that is beside the point).

Finally, and maybe most importantly, CEU should pursue all legal means against the Hungarian Government to get compensation for our damages. Recouping some of the expenses of this move will definitely help in putting the University back on a more solid financial footing. (This was costly. While our less than stellar financial performance pre-date the crisis, it is not unrelated to the costs endured with the move either.) I am not a lawyer. I don’t know how this works. But some avenue must be available to, at least, financially remedy the wrongs done to CEU. Sure, Fidesz will tell the Hungarian people that we are stealing their money, but that is not true. They stole the Hungarian people’s money when they, defying all rule of law, kicked CEU out of the country. Taking legal action against the Hungarian government will certainly make our relationship to Hungary more contentious. If that happens, we would need to worry about the safety of our real-estate, the safety of our staff and the safety of our students (something that has not been an issue so far). I don’t believe we could take legal action while we operate a campus in Budapest.

Of course, there are arguments for maintaining a strong presence in Hungary as well. The academic arguments are strong. At times very strong. Take for example the Baby Lab tied to the Cognitive Science department who run developmental panel studies of many many children, all in Hungary. Such academic operations need not be shut down, but they also do not require a whole campus. (Considering the Baby Lab used to operate out of a completely different building on the other side of the Danube for years, I do not think this argument is controversial.) Multiple departments have a closer tie to Hungary and CEE in general. I understand that they worry about moving out of this region. But it is not like we have much of a choice here. With faculty replacements down the road, these units will diversify. In the mean time, I am sure there is no doubt in anyone’s mind that if one wants to study what they focus on, CEU is the place to be in Budapest or Vienna. I would argue that we should not leave our academic roots in Hungary, we should, in fact, up our Hungarian societal outreach. We should build on the many Hungarians on CEU’s faculty to do this. Research on Hungary should persist. We should remain parts of the academic networks. I certainly hope to keep my position on the board of the Hungarian Political Science Association. I hope Zsolt Enyedi (the person who, if he got this far, is probably quite unhappy with me right now) and I will continue to collect public opinion data after each parliamentary election to ensure we stay part of the international network of election studies that make great electoral comparative research possible. But we do not need to keep our old campus to do this. Such activities, in most cases, can be coordinated out of a modest office and in most cases directly out of the Vienna campus.

What pains me the most in writing this is realizing how much our friends and colleagues, the social scientists at Hungarian universities and research institutes will lose if we move the library away. Whatever office is left of CEU should include a reading room, access to all of our electronic resources and a way to request book transfers. This is the least we can do.

Our Hungarian accreditation is up in 2022. The University is already trying to figure out how we could keep enough Hungarian employees to maintain Hungarian accreditation, but also have these people teach in Vienna. Any outcome of this is hardly an ideal scheme. Our romantic ties to Budapest will fade with time. New faculty will have no connection to Budapest. What will remain is  CEU that we all want to see successful both academically, financially and as the beacon of freedom where the future leaders, societal investors and social scientists are trained. Let’s make sure that we make the right decisions now to ensure we can continue doing what we do, in Vienna, under the best of conditions.

 

Thoughts on CEU: What I lose with the Vienna move.

In my previous blog post on how to generate more revenue for CEU than a tuition scheme ever could (and do so without selling out our principle), I promised that this is only the beginning of a series. I also mentioned that I will, soon, write more about how “the CEU2025 plan […] (with generous help from the Hungarian government) will destroy everything that made CEU a great place to work for me.” This is going to be a very personal post and, really, nobody should care about this (in fact, if you have not done so yet, read the previous post even instead of this one). But I do want to lay out, in print, what gloomy future I see in the current transformations and our CEU2025 plan. This post is about how all these things affect me and only me. Others’ opinions may vary.

Let’s take stock at what CEU offered me. It allowed me to (1) be at home, in Budapest (in one of the nicest parts of the city), while (2) working in a mostly American academic environment, at a (3) prestigious institution where (4) we only have to work with the most ambitious students who decided to get a graduate degree. (5) We have a manageable teaching load in line with the research focus of the institution. We have time to do research. CEU (6) offered us a salary that is kind of sort of competitive with the Western world (where all of us coming to CEU – at least in the past decade) would have been looking for jobs) that, factoring in cost of living, allowed for an upper-middle class life. So let’s take this point by point in no particular order.

Budapest (1): CEU is leaving Budapest for the most part. A commute is impractical, to say the least. My wife and I decided not to do it. CEU will be going through incredible changes. I do want to be present to shape this change. But this comes at a cost. I am losing the closeness of my family and my home. I worked hard to move back from the US after 10 years of college there. Now I leave not by my own choosing. (I am not the only one who left the country because of a lack of appropriate jobs around that pay anything, so my personal tragedy is hardly unique. But this still feels a little different than economic migration.) Personally, my living arrangements are (were) ideal in Budapest as I live in a separate and individual apartment in a larger housing unit that my parents built for our entire family. Not counting a short pause between the death of my great grandmother (at a 101 years of age) in 2009 and the birth of my daughter in 2016, we have been a four generation household for most of my adult life. With the Vienna move I will have to leave this (or commute which, while considered perfectly normal for academic families in Germany, is brutal for someone who has different conceptions of normal like myself). While my rent currently is practically 0 I did recently sink the price of a small downtown apartment into fixing up the place, finishing an entire unfinished floor. (Something I will never get back.) Now my rent goes from this to… whatever it is going to. (Probably around 1500 EUR.) I have no equity in the existing real-estate. I only had the renovation expenses I did to make a life for my family there. I guess I will always have a nice place to come back to if I ever want to spend a weekend or a few weeks in Budapest. I also lose free, instant and on demand child care. While others may prefer not having their family so close, losing that for me is incredibly difficult.

Graduate Only (4): To become a university in Austria, we had to accredit two undergraduate programs (and I understand, as I am part of the team putting it together, plans for more are on the way). One of these two programs is PPE (Politics, Philosophy and Economics) where my department, the department of political science contributes greatly. So we will now have to teach undergraduate students. During my graduate work at the University of Nebraska-Lincoln, between 2002 and 2006, I taught undergraduates in the US. On average, this was a course a semester (or, while I was still an MA student, equivalent number of discussion sections for large lectures of freshman level Intro to American Politics or Intro to International Relations offered by a professor). My own courses included Public Issues in American Politics; Elections, Parties and Interest Groups; and Polls, Politics and Public Opinion. During my temporary appointments at CEU (2007-2008) I taught a course a semester at ELTE. Courses included Intro to Political Science and an adoption of the Public Issues class I taught in the US. I enjoyed teaching undergrads at the time, but to do this well is a lot of work. I don’t feel I have the experience to do this anymore. Working with graduate students, the expectations of an instructor change. I am sure I could get back into it, but it will be a transition and, let’s face it, from the perspective of prestige, it will be a downgrade. In sum, this is not ideal but teaching undergraduates is certainly not the end of the world.

The teaching load (5): This I worry about. As we move to an institution that is all graduate school and research to one that also focuses on undergraduate education, the teaching load may also increase. This is especially true in light of the financial position of the university (which I wrote about last time). Most of the plans circulating wish to add the undergraduate education without increasing the number of professors or pushing the undergraduate teaching on PhD students (which would not be a bad idea in my opinion). Some suggestions included the restructuring of PhD education in a way to eliminate teaching mostly or entirely. Either way, I fear, the teaching load will have to increase or the American Character (2) of the program that is strongly focused on teaching our PhD students will suffer. I don’t want to make this post about the relative merits of having or not having PhD courses, but let’s just say I believe a shift here would affect the political science program negatively and I personally don’t like it. In other respects we will continue to work as an American university. Most of this we certainly will not lose. In fact, if the parts of the University that were the most “Hungarian” (and I don’t just mean the canteen) could become a little less “Hungarian” and that would be an upgrade.

Salary (6): CEU proposed to raise salaries of assistant professors 25%, Associates 20% and Full 15%. (I fall in the latter category due to my very recent promotion. I even wondered if asking for promotion now was a mistake. But I think CEU would have managed to even it all out with the raise I got (or would have gotten next year) with the promotion. So I am going to believe it is a wash.) Austrian taxes are quite a bit higher than Hungarian taxes (so are the services, I guess – this certainly is a plus). But the cost of living adjustment is 50%. Housing price differential is 80% (which, as you see above will hit me particularly hard). Given my personal situation, I do not think I can maintain my current quality of life short of a 100% raise. And that I will certainly not get.

The University leadership is framing this issue the following way. We are told that Austrian academic salaries are actually comparable to (as in, not much higher than) our current salaries, hence the modest raise. Of course, anyone who knows anyone at the University of Vienna knows that these numbers are only half-truths. They are salary floors, possibly the ones in the union contracts. The reality is, in the words of one University of Vienna colleague, these sums are only made by people who have never been in a competitive situation in their lives. They got their PhD here and moved up the ranks. They never negotiated. Nobody at CEU got their professorships this way. The person also added that “I make a heck of a lot more than those numbers”. The fact is that actual salaries, especially of competitive people we believe we are, and especially in Vienna, are quite a bit higher. I am going to give leadership the benefit of the doubt that they simply do not know this. But they are also in a difficult position. (See the previous post on CEU finances.) But the framing still bothers me. Speaking of frames, I also heard the following pitch from the CEU Vienna office. If I had the opportunity to go teach at Columbia, could I still expect the school to provide my CEU quality of life where I can easily live in the best area of Budapest (so best area of Manhattan)? Of course not. (When I heard this pitch in the Vienna transition office, I was not there to complain about salaries and I have done nothing of this sort. I was asking if they know about the specific registration times for schools because I do not want to get there and not have access to the public education system. I don’t know if it is muscle memory, but I still got this pitch. I am going to let everyone figure out what is wrong with this analogy. The list could have its own post. And, sure, fair enough. Let’s accept that Columbia frame.)

I am not here to complain about my future salary, but there is a lot to complain about. I care about CEU’s financial health so I am holding myself back here. But I would be lying if I said this is not an important item on that list of things made worse by the transition to Vienna. And this will come at other costs. While I may be committed to CEU, not everyone will be. Some of the best people will certainly leave. With these salaries, with the higher Austrian cost of living, with the undergrads our ability to recruit the best faculty to replace them will certainly diminish. The quality of research output will go down. We will have a harder time competing for grant money also because we are not anymore in that difficult region of post-communist CEE where you want collaborators from on your team but often have a difficult time finding people who can maximally contribute. CEU was always an obvious choice. All of this means rankings will go down, that means we will have a more difficult time recruiting the best scholars for our grad programs, best researchers for our PhD (and post-docs) and that means we will have an even more difficult time recruiting the best faculty and the cycle perpetuates with the potential to seriously hurt the prestige (3) of CEU. In essence, we need to put a break on this downward spiral that the university is certainly in danger of getting into. Without financial health, we have little to no abilities to do this. (This is why I wrote the first post in this series and this is why that was the first. But watch out for more posts on the subject.) I can say this much, I am giving CEU many years to show that this pessimistic outlook of the future is not our path. And I will work every moment of every day to put breaks on this downward spiral before I am going to let my commitment to CEU wane. And I know that there are many people who think the same way. This is why CEU will continue to be a great place to work, a great place for students and a great place to be, in general.

Thoughts on CEU: Would Income Share Agreements Work for Us?

CEU is going through an incredible transformation. We are moving countries and we are facing new challenges. CEU leadership laid out a plan that is referred to as CEU2025. I plan to write about what I am about to say in an itemized list but my broad conclusion after looking at the CEU2025 plan is that it (with generous help from the Hungarian government) will destroy everything that made CEU a great place to work for me. (Others’ opinions may vary, but as far as I am concerned, this is how I see it.)

I have made a promise to myself that, despite more regular solicitations, I will stay at CEU and see all this through. I will try to contribute to the best of my ability. But in the process, my main goal will be to ensure this negative foreshadow of the CEU2025 future does not come to fruition. Next fall I will give up the on-demand baby sitting just one floor below where my parents live and move to Vienna because I believe that will be the best place where I can contribute to making an Austrian CEU an even greater place to be both for us, the faculty, and, most importantly, for our students.

These blog posts under the “Thoughts on CEU” series will be in this spirit. Hopefully they can contribute to a conversation on how CEU could thrive despite its new challenges.

To begin the series, we have to talk about something none of us want to talk about. CEU, despite recent recapitalization of the endowment and additional money for the Vienna transition (and maybe also cash for a permanent campus in Vienna), is not in good financial shape. It is fair to say most people on campus are not honest about this fact. Even the ones who may have been concerned before are now fooling themselves with this recapitalization. We, the faculty, do not have on demand access to accurate and up to date information about our yields, endowments, budget, but I can cite a conversation with the Rector here (from before our “little local troubles” started) we are not in good shape and the current influx of capital only puts us on life support, giving us a chance to get back on our feet and not be doomed in the long run.

To combat the problem, CEU2025 proposes to collect 3000 EUR tuition even from people who previously received a full ride. (Note that CEU2025 was only a few slides and the proposals are, of course, fluid. That is exactly why I am putting these thoughts out in the open.) Above, I mentioned that I have a long list that makes CEU a great place to be (and in my next post I will itemize). Here’s the first one: we have excellent and ambitious students many of whom had no opportunities elsewhere because they cannot afford it.

This tuition move puts CEU outside of the reach of some of our best students. Cost of living increases in Budapest (mostly due to steeply increasing real-estate prices) of the past 6-8 years have already put a strain on low income families’ ability to send their kids (and sometimes spouses, mothers and fathers) to CEU. Add Vienna cost of living ,which is around 50% more than in Budapest (with housing prices 80% higher) along with this hint of 3000 EUR tuition certainly puts us out of the reach of our traditional demographic of students who cannot afford a Western education. CEU, traditionally, was the jumping board to a Western education for much of the post-communist region. By the time I joined CEU in 2006, the West was directly accessible to many, but not to everyone. If we lose this demographic, I wonder who will come to CEU? This, of course, calls for a longer conversation but I do not believe that the appropriate market research has been done to answer it. If it has been done, I haven’t seen it. So, for now, lets just say that if we trade ambitious people who had no other opportunities for well off people who had no other opportunities, CEU’s reputation will certainly suffer (not to mention its academic staff).

But maybe there is an appropriate way to handle this without charging tuition. A few months ago a 50+ year old idea dating back to Milton Friedman’s book Capitalism and Freedom received quite a bit of attention in the news: Income Share Agreements. Agreements that a student can attend a University for free and in exchange they will share a certain percentage of their income for a certain amount of years. NPR’s Planet Money described it as a University buying stock in a student they train. They literally financially invest in the student’s future success.

Now, that sounds like CEU to me. We pride ourselves in our students, in training the future leaders of the entire region. We have students who are mayors of capitals, ministers, EU administrators. We believe in our students. So instead of charging them tuition, why not invest in them?

There are many nuances of how these income share agreements work to go through in such a blog post. But let me highlight some of the most important ones. CEU is not trying to generate that much revenue per student. So we could keep the income share quite low. Purdue University’s program asked 15% for 8 years from someone who worked in food science. (Purdue University also varied the percent depending on the person’s major, but this is a non-issue at CEU.) We can probably do much better keeping the percent at a non-scary number. Say, 5% would sound OK to me. Our tuition are nowhere near Purdue’s (though maybe neither is the projected income of our students, not sure). I guess we can keep the time longer if need be. The 8 years was 8 years while a person was in employment or was seeking employment. If someone wanted to get out of the labor force and make money traveling around the world busking, that did not count. (They could do it and pause their 8 years.) On the other hand if someone was laid off and was searching for their next job, that counted towards the 8 years. There were protections in place for the students. If they became mega-millionaires. They still only had to pay back 2.5x the cost of their tuition (a tuition that, with such a program in place, CEU could totally raise without a bad conscience). The amount the school expected back was capped.

Such an arrangement would allow our historical demographic access to CEU, especially if CEU provided some need based housing and scholarships to offset Vienna’s relatively high cost of living. It would put CEU in the world of cutting edge outside the box thinking, innovative solutions true to who we are and who we inspire to be. And there are other benefits. First, CEU received strong criticism for its “neo-liberal” behavior from an internal activist group. While I share the group’s concerns, I worry about their numeric literacy. If their goal was to just blow the endowment over the next decade or two, then I do not share these goals. If they thought figuring the numbers out is not their job, I do not find their propositions constructive. Maybe they were just, like so many at CEU, ignorant about our financial situation. This strategy is one that even such a radical group and the people whose job is to look after the financial health of the University could (maybe even should) agree on. Also, let’s say such a program is launched and the 10 or so years (whatever it will be) is up for the first students. The University, at this time, will have a strong personal but also contractual relationship with this student. We would have a great ability to communicate with them, find them if we lose track of them (because we would have the ability to collect more information from them even in the age of GDPR and use this information for staying in contact). So why not ask them to keep contributing? Not everyone will, but in the spirit of CEU’s mission, they may just chose to target CEU (or its future students) with their philanthropy. They are used to paying 5% of their income to CEU. Why not just keep doing that (and get a tax write off)? Maybe these people will be the ones who put CEU in their will and 75 years down the road, the benefits will multiply.

There is just one major complication this is all pointing to. In the past I have often asked why CEU doesn’t offer student loans for tuition or housing ourselves. The response was always, we simply cannot. We don’t have the ability to track students from a 130 countries to get our money back. That is true. But you know who else doesn’t have that ability? NOBODY! None of the financial institutions operate in all countries effectively. Some may decide to take on such a task but it will cost us, or it will cost the student dearly with heavy interest. The reality is that CEU is in the best position to administer such a program. We are used to offering education, even for free to students. Let’s keep doing that with deferred income. We do not have the ability to go after anyone in every corner of the world, but we have the ability to put a list on our website of the former students who defaulted on their obligations. (Not something we should take lightly, but this ability is a stronger enforcement mechanism than what anyone will have. One’s reputation is important.) Will some people default? Sure. Will it be such a large percent to declare the project a failure? Maybe I have too much faith in people but I sincerely do not believe so. I know this is an experiment, but what we lose if we, in the spirit of CEU2025, start charging tuition of everyone (or just about) is quite clear. The potential for gain with this alternative structure may be greater than anyone could predict today outperforming any tuition scheme we may put in place. So why not try it? If it fails completely, we can go back to a more traditional structure.

NOTE: Many of the ideas presented here come from the above linked NPR Planet Money Podcast and the Freakonomics interview with Purdue President (and almost Republican US Presidential contender) Mitch Daniels.