Studies

1. Polack et al. (2020). Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine, 383, 2603-2615. → This is the original phase 3 clinical trial of the Pfizer/BioNTech vaccine. It included a total of 44,000 participants and showed an efficacy of 95%, i.e. there were 20x more Covid infections among the unvaccinated control group than among the vaccinated group.

2. Dagan et al. (2021). BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting. New England Journal of Medicine, 384(15), 1412–1423. → This is an even larger study, conducted in January in Israel. Based on the data of 1.2 million members of Israel's largest health insurance provider, it showed that vaccinated people were 10x less likely to contract Covid, be hospitalised with it, require intensive treatment, or die (i.e. a 90% efficacy).

3. Sheikh et al. (2021). SARS-CoV-2 Delta VOC in Scotland: Demographics, risk of hospital admission, and vaccine effectiveness. The Lancet, 397(10293), 2461–2462. → This is a study of 20,000 infections in Scotland, looking at the Pfizer/BioNTech and the AstraZeneca vaccine and contrasting their effectiveness against the Alpha variant and the (currently-dominant) Delta variant. For Pfizer, they found 90% and 80% efficacy against Alpha and Delta, respectively; for AstraZeneca the numbers were 70% and 60%.

4. Henry et al. (2021). Effectiveness of COVID‐19 vaccines: Findings from real world studies. Medical Journal of Australia, 215(4), 149. → This is a review study that combined and compared the results of five different previous studies. It showed that the efficacy varies with various factors such as the vaccine used or the time since vaccination, but seems to consistently lie within a range of 60-90% for fully vaccinated persons.

5. Hsu et al. (2021). COVID-19 Breakthrough Infections and Transmission Risk: Real-World Data Analyses from Germany’s Largest Public Health Department (Cologne). Vaccines, 9(11), 1267. → This is a study by the municipal health department of the city of Cologne, who are responsible for contact tracing and quarantining of infected persons and their close contacts. Here, they show that unvaccinated contact persons of infected vaccinated persons had a 15% chance of becoming infected, as compared to unvaccinated contact persons of infected unvaccinated persons, who had a 65% rate of infection. This shows that even in the case of break-through infections (i.e. when vaccinated persons do become infected), they are much less likely to pass that infection on to others.

6. de Gier et al. (2021). Vaccine effectiveness against SARS-CoV-2 transmission to household contacts during dominance of Delta variant (B.1.617.2), the Netherlands, August to September 2021. Eurosurveillance, 26(44). → This study agglomerates contact tracing data from municipal health departments across the Netherlands during a period of Delta dominance. It shows that although 71% of the population were vaccinated, only 35% of reported infected cases were. It also showed that while unvaccinated household contacts of unvaccinated infected persons had a 22% chance of infection, that was halved to 10-13% if either the originally infected person or the household contact were vaccinated.

7. Martínez-Baz et al. (2021). Product-specific COVID-19 vaccine effectiveness against secondary infection in close contacts, Navarre, Spain, April to August 2021. Eurosurveillance, 26(39). → This is a similar study to the two above (based on contact-tracing), this time from Spain. Here, too, the authors found a 50-80% reduction in infections among vaccinated persons, depending on the vaccine given (though quite strongly reduced for elderly persons).

8. Bar-On, Y. M., Goldberg, Y., Mandel, M., Bodenheimer, O., Freedman, L., Kalkstein, N., Mizrahi, B., Alroy-Preis, S., Ash, N., Milo, R., & Huppert, A. (2021). Protection of BNT162b2 Vaccine Booster against Covid-19 in Israel. New England Journal of Medicine, 385(15), 1393–1400. → Another study from Israel, this time looking at how long the protection from the second vaccination lasts and how effective a third “booster” vaccine is. They find that 5-6 months after the first two shots, the vaccine's efficacy has been strongly reduced, but can be restored with a third shot. (Side note: there are vaccines, like that against Hepatitis B, that require three shots to reach their full effect. There are reasons to believe that the Covid vaccines might be like that too, i.e. that we will not require regular boosters but be sufficiently immune for the long term after three shots. We don't yet have enough data to say for sure, but we will find out in the next few months.)

9. Maier et al. (2021). Germany’s current COVID-19 crisis is mainly driven by the unvaccinated. ArXiv:2111.12806. → This study is not yet officially published and has yet to undergo peer review, so should be treated cautiously until it is. (All scientific studies are thoroughly reviewed by independent experts, and the authors have to address any queries or critiques the reviewers have before publication.) However, if the results hold up under review, they indicate that only 10-15% of infections in Germany are happening between two vaccinated persons, while 40-50% of infections are taking place between two unvaccinated persons.

10. Bavarian State Ministry of Health (see image below) → This recent graph from the data of our state's MoH shows that the relative infection rate (i.e. infections per 100,000 citizens) in November (when I downloaded the graph) was much lower among vaccinated persons than among unvaccinated persons - 110 vs 1500! The rate of hospitalisation was also significantly lower - 3 vs 14 per 100,000. (Note: this is not at all a formal scientific study, but a neat illustration of the very real effects vaccination is having in preventing infections and illness of individuals at a time of generally very high infection levels in Germany.)

Closing comments

I have had long discussions with a number of friends here who are sceptical about the vaccines, and have read numerous websites that they passed along to me. None of them stood up to closer scrutiny. In many cases, they simply offered unsubstantiated speculations about possible long-term effects of the vaccines (which, for a whole range of biological reasons, are highly implausible). In many more cases, they grossly misinterpreted and/or misrepresented scientific data and studies (and yes, I did read the original studies to check).

Yes, epidemiology is really complicated. Who gets infected and/or seriously ill is influenced by any number of factors, including things like current infection levels in society, age, sex, vaccination status, medical risk factors, personal contacts, weather, public hygiene measures, and others. Likewise, vaccine efficacy and safety are also affected by numerous factors, which makes data analysis tricky and requires some very careful scientific work. But this work has been done.

Nothing in this life is ever one-hundred-percent effective, or one-hundred-percent safe. Neither are the vaccines. But after hundreds of millions of vaccinated persons around the world and with the accumulated knowledge of dozens of large, careful scientific studies, we know enough to say: vaccines work, and it's a lot (!) safer to be vaccinated than to risk contracting Covid.

Put like that, it is a bit of a tongue-twister :D But quite honestly, this is no joking matter - and not as difficult as it sounds. The first thing you'll be taught in any epidemiology class (and yes, I have taken a couple) is that during an epidemic, there are three basic classes that people fit into: Susceptible, Infectious, or Recovered. (This SIR-model is simplified, of course, but works quite well nonetheless.)

The idea is that infectious people pass on the disease to susceptible people around them, but, once they have recovered, are immune to being infected a second time. This means that over time, there are more and more recovered/immune people in the population, so that the people currently infected are less and less likely to meet susceptible people and eventually the disease dies out.

The problem is of course that the process of you being sick may lead to you recovering and becoming immune, but it may also lead to you being very seriously sick or even dying. So the question is: how do we break the cycle of infection? And there are two ways of doing that.

The first is what we were doing all of last year: social distancing, masking, quarantining, etc. Not a nice route to go, but it helps to prevent infectious people from meeting susceptible people, thus slowing the rate of infection and saving lives.

The second way is vaccinating. Vaccines simulate the sickness process, but with two crucial differences: (1) you are much less likely to suffer severe health consequences, and (2) you are not completely guaranteed to become immune.

Basically, getting the jab “jumps” you from the “susceptible” class into the “recovered” class, without having to be properly sick along the way. Unfortunately, this only works with a probability of 60-95% (depending on the vaccine [1]). So while this is a very good deal compared with the risks of actually contracting Covid, it isn't bullet-proof.

But it doesn't have to be. Because a disease will die out if the percentage of susceptible people in the population is low enough, you don't actually need full protection for every individual to eradicate an epidemic.

Imagine a town with 10 000 inhabitants. If they were all vaccinated with a vaccine as effective as Pfizer or Moderna, about 9500 citizens would become immune. And although a random sample of about 500 would still be susceptible, that wouldn't matter much, because there are too few of them to keep the disease spreading.

But now imagine that only half the citizens were vaccinated. The unvaccinated people would keep getting sick, keeping up a high rate of infection and so quite probably exposing the 250 people who were vaccinated, but for whom the vaccine did not work as intended, to the disease.

It is exactly this effect that is the problem. The Covid vaccines we have work remarkably well, and have been shown to do so in every country that is using them [2]. But they are still not perfect. And because of that, it does make a difference how many people are vaccinated overall.

In short: if you haven't been vaccinated yet, you are raising the risk of others around you - including some who have already been vaccinated. So get your vaccine and help to save a life ;-)

Addendum 1: But vaccinated people still get sick, and then infect a lot more people, because they don't have symptoms and don't realise they are infected!

Vaccinated people are much less likely to become infected at all [2], and if they're not infected, they can't pass it on. It is true that there are cases of unsymptomatic infections in vaccinated persons, but these are (a) comparatively rare, and (b) current evidence indicates that unsymptomatic cases are probably not as infectious as symptomatic cases [3]. Also, it's not that easy to quantify how much somebody is at risk - although there are some clear risk indicators, that doesn't mean that healthy young adults cannot get sick, and badly so [4].

Addendum 2: The government shouldn't force you to get vaccinated, it's the individual's choice!

Whether or not it should be mandatory is a whole other question. (Although note that mandatory vaccinations in and of themselves are nothing new.) But I do find it important to emphasise that the choice to be vaccinated is not just an issue of personal preference, but a deeply ethical question of responsibility to others. In that sense, getting the jab is a form of loving your neighbour ;-)

Addendum 3: But natural immunity is so much more effective than vaccination! [5]

That's an interesting study! However, I don't quite understand why you see natural immunity as an alternative to vaccination? Basically, what the study says is: “If you were lucky enough to survive Covid once, you're almost certainly safe in the foreseeable future”. But of course the point of being vaccinated is that you don't have to play the Russian roulette that is a Covid illness. The study investigates ~100 000 Covid survivors - but doesn't mention that ~16 000 of them would have been quite seriously ill, or the ~2000 additional infected people who didn't make into the study cohort because they died of their illness (numbers based on hospitalisation and mortality data from the Israeli Ministry of Health [6]). Compared to the known short- and long-term risks of a Covid infection, a vaccine may not be 100% safe (few things in life are), but the overall risk is lower by several orders of magnitude. And, at the risk of repeating myself, that is before we consider the effect that my vaccination choice has on the safety of those around me.

Sources

1. European Medicines Agency. “COVID-19 vaccines”.

2. e.g. Dagan et al. (2021). “BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting”. The New England Journal of Medicine.

3. Mallapaty (2021). “Can COVID vaccines stop transmission? Scientists race to find answers”. Nature News.

4. Robert Koch Institut, Epidemiologischer Steckbrief zu SARS-CoV-2 und COVID-19.

5. Gazit et al. (preprint). “Comparing SARS-CoV-2 natural immunity to vaccine-induced immunity: reinfections versus breakthrough infections.” medRxiv.

6. Israeli Ministry of Health, Covid-19 Frequently Asked Questions

Also, fangen wir vorne an: Zu einer wissenschaftlichen Arbeit gehört, dass man sie in den Kontext der aktuellen Forschung einbettet. Zu Beginn gibt man einen Überblick über neue theoretische und empirische Erkenntnis und skizziert offene Fragen. Dann erklärt und begründet man die eigene Methodik und beschreibt die erzielten Ergebnisse. Schließlich interpretiert man diese Ergebnisse mit Blick auf den Stand der Forschung und diskutiert, welche Auswirkungen daraus folgen könnten. Ein durchschnittlicher Fachartikel zitiert deswegen etwa 50 andere Studien, die es dem Leser erlauben, sich selber einen Überblick über den Hintergrund der Arbeit zu verschaffen. Seligmann zitiert genau 0 Fachartikel – das ist einfach nur unseriös.

Aber das ist noch lange nicht das Hauptproblem. Kurz gesagt hat Seligmann schlicht keine Ahnung von Statistik, er ist hier methodisch völlig außerhalb seines Kompetenzbereichs. Um das zu erklären, muss ich etwas weiter ausholen. Statistik ist die Wissenschaft der Datenanalyse, oder vereinfacht ausgedrückt, eine Sammlung von Methoden, um aus rohen Daten wissenschaftliche Aussagen zu gewinnen. Nicht jede Disziplin ist gleichermaßen auf Statistik angewiesen. Seligmann ist Molekularbiologe – diese arbeiten meist unter streng kontrollierten Bedingungen im Labor und brauchen daher oft nur sehr rudimentäre statistische Methoden. Andere Forschungsbereiche, wie Ökologie (meine eigene Disziplin) oder Soziologie haben da ganz andere Anforderungen. Dadurch, dass Ökosysteme und menschliche Gesellschaften so unglaublich kompliziert sind, von Tausenden von Faktoren beeinflusst werden und Experimente mit ihnen nur begrenzt möglich sind, müssen wir sehr aufpassen, wie wir unsere Daten analysieren. Deshalb brauchen wir oft sehr komplizierte statistische Methoden, die dieser Komplexität des Lebens zumindest ansatzweise gerecht werden können.

Seligmann begeht hier den fundamentalen Fehler, seine sehr einfachen Methoden (Pearson-Koeffizienten und t-Tests) auf Daten anzuwenden, die viel zu komplex dafür sind. Obwohl Mortalität von beliebig vielen Faktoren beeinflusst wird (Hitzewellen, Grippewellen, Sportereignisse, Verkehrsaufkommen, Lockdowns…) ignoriert er alle außer der Impfrate. Das darf man aber nicht so einfach machen: Es könnte ja sein, dass zwar die Mortalität tatsächlich mit der Impfrate steigt (was seine Daten nicht hergeben!), dass aber nicht das Impfen das Problem ist, sondern das steigende Verkehrsaufkommen, wenn die Menschen wieder aus dem Homeoffice kommen.

Damit ignoriert Seligmann die wichtigste Regel, die es in der Statistik überhaupt gibt: „Correlation does not imply causation!“ Das ist wissenschaftlich unentschuldbar. Aber es geht weiter. Zwar berechnet er die Signifikanz seiner Zwischenergebnisse, also die Wahrscheinlichkeit, dass solche Ergebnisse einfach nur zufällig zustande kommen. Doch für seine nächsten Analyseschritte ignoriert er die Signifikanz völlig und rechnet fröhlich mit Zahlen weiter, die so unsicher sind, dass sie genauso gut null sein könnten.

Fazit: Beide seiner Analyseschritte sind von Grund auf völlig falsch. Damit ist es absolut egal, was für Zahlen am Ende seiner Rechnung stehen – sie können keinerlei Aussagekraft haben. Doch selbst damit nicht genug: In seiner Diskussion versteigt er sich schließlich in die abstrusesten Spekulationen. Was er hier schreibt, ist in keinster Weise von sonstigen wissenschaftlichen Erkenntnissen gedeckt, noch wäre es gerechtfertigt, selbst wenn seine Analysen richtig gewesen wären. Spätestens hier hat er für mich die Grenze der wissenschaftlichen Inkompetenz überschritten: ein solches Gefasel als „wissenschaftlich“ zu verkaufen ist einfach nur unredlich.

Siehe auch die Faktenchecks von Mimikama und Correctiv.

Well, you just sent me down the rabbit hole with that one… The quick summary: the post above is misleading, but not completely wrong; the evidence for mask effectiveness is spotty, mixed, but more positive than not.

I've taken a quick but not detailed look at the studies the author cites, as well as reviewing around a dozen other studies. I also have some general comments about the nature of scientific evidence. (Yes, this is going to be a long answer ;-) )

Let's start with the latter: What the FOIA request was asking about was something very specific: randomised control trials (RCT) of mask-wearing with Covid-19.

RCT are a type of experimental design whereby you take a number of test subjects, randomly assign them to a treatment or non-treatment (i.e. control) group, and then compare your response variable across the groups (here: rate of Covid-19 infection). When done properly and with a sufficiently large number of subjects (the “N”), RCT are the gold standard of medical study design. However, they can pose problems. For a start, they take a lot of effort, time, and money to implement. Especially because you really do need a large N, if possible in the thousands - an RCT with a low N is worse than useless. Also, there may be ethical concerns: if you have a risk-free treatment for which there is already some indication that it works, it would be problematic to refuse to give this treatment to your control group - potentially risking their lives just to get slightly better data.

So, for this combination of practical and ethical reasons, it is understandable - though very unfortunate - that there is not yet a RCT of masks with Covid-19. However, RCTs are far from the only study design available to researchers. Two other common categories, which have been used to address masking with Covid-19, are observational and laboratory studies.

Observational studies are what are known as “quasi-experimental” designs. The idea is to find a situation which is as close as possible to the experiment you'd like to have carried out, and then analyse the participants in this situation. For example, there was a study of Covid-infection on flights, where the passengers on some planes had worn masks while those on other planes hadn't. Or a German city that introduced compulsory masking several weeks before the rest of the country. This has multiple advantages: 1) you don't need to spend time conducting the experiment, the situation has already occurred; 2) it's often easier to get a large N; 3) there are no ethical concerns because you aren't actively intervening in the situation. The major drawback is that you can't exclude confounding factors the way you can in an experiment. Indeed, as these are real-life situations, there are likely to be a lot of factors at play, whatever the outcome. There are statistical techniques for reducing the uncertainty this introduces, but basically you'll never be able to guarantee that there wasn't some other factor to blame for your results.

Laboratory studies, on the other hand, exclude “real-life” as much as possible, only recreating the bare necessities in a completely controlled environment. For example, one study used two dummy heads in a glass cage, with infectious aerosols being sprayed out of the one dummy's mouth and sucked in and measured through the other's mouth, with different combinations of masks in between. The benefits of this are that you need few or no human subjects and can completely exclude any factors you're not interested in. The problem is that those excluded factors may actually be the more important ones, or you may be asking the entirely wrong question. (“Yes, masks can physically stop virus particles if they are worn in public - but what if most infections are happening at home?”)

In summary, there are different study designs, all of which have merits and all of which have flaws. While RCT are deservedly seen as the best, it is not appropriate to just ignore all the others, or claim that they are “bad science”.

Among the studies I looked at, there were observational and laboratory studies on masking and Covid-19, as well as RCT on masking and other viral disease (which also should not be discounted!). The evidence is not as clear as I would have expected, and there are not as many studies as I would have thought. Among the studies I found, some say there is no effect, most say the effect varies or is very slight, while yet others find a clear effect. As always, some studies seemed to have a stronger design and/or analysis than others.

In general, my impression was that overall, masks do help - but they are not a cure-all, and their effectiveness is obviously limited to the occasions when they are actually worn. (I.e., masking rules may reduce infections in public gatherings, but they won't stop infections in private home settings.)

The complete list of studies I used is below. The one I thought the best is Chu et al. (2020). This is a meta-study, so it combines the results of many individual studies.

Having already said all that, I should emphasise that situations like this, with spotty and equivocal data even on important questions, is actually very common in science. I'm planning to do my PhD in conservation biology on the question of how effective protected areas are at protecting biodiversity. The first PAs were established 150 years ago, we've been using them intensely for 50 years, and we're still not sure how much they actually help. Turns out, collecting data is really hard and expensive, and nature is incredibly messy. (I'll be combining data sets from over 20 different organisations and agencies, and the analysis techniques I need to use are partly still being developed.)

So in that sense, I'm not particularly worried by the uncertainty in the masking question. We know PAs help at least somewhat, so we keep lobbying for them. If we didn't do anything until we could put a precise percentage on their effectiveness, there would be no nature left to protect. Likewise, we know there can be positive effects of masks, and the negative effects are minor. So if the situation is urgent (which it is) and every little counts (which it does), I think it would be foolish to ignore them, even if we don't yet know as much as we'd like.

Bibliography

• Bundgaard, H., Bundgaard, J. S., Raaschou-Pedersen, D. E. T., von Buchwald, C., Todsen, T., Norsk, J. B., Pries-Heje, M. M., Vissing, C. R., Nielsen, P. B., Winsløw, U. C., Fogh, K., Hasselbalch, R., Kristensen, J. H., Ringgaard, A., Porsborg Andersen, M., Goecke, N. B., Trebbien, R., Skovgaard, K., Benfield, T., … Iversen, K. (2020). Effectiveness of Adding a Mask Recommendation to Other Public Health Measures to Prevent SARS-CoV-2 Infection in Danish Mask Wearers. Annals of Internal Medicine, 174(3), 335–343. https://doi.org/10.7326/M20-6817

• Chu, D. K., Akl, E. A., Duda, S., Solo, K., Yaacoub, S., Schünemann, H. J., Chu, D. K., Akl, E. A., El-harakeh, A., Bognanni, A., Lotfi, T., Loeb, M., Hajizadeh, A., Bak, A., Izcovich, A., Cuello-Garcia, C. A., Chen, C., Harris, D. J., Borowiack, E., … Schünemann, H. J. (2020). Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: A systematic review and meta-analysis. The Lancet, 395(10242), 1973–1987. https://doi.org/10.1016/S0140-6736(20)31142-9

• Cowling, B. J., Zhou, Y., Ip, D. K. M., Leung, G. M., & Aiello, A. E. (2010). Face masks to prevent transmission of influenza virus: A systematic review. Epidemiology & Infection, 138(4), 449–456. https://doi.org/10.1017/S0950268809991658

• Doung-Ngern, P., Suphanchaimat, R., Panjangampatthana, A., Janekrongtham, C., Ruampoom, D., Daochaeng, N., Eungkanit, N., Pisitpayat, N., Srisong, N., Yasopa, O., Plernprom, P., Promduangsi, P., Kumphon, P., Suangtho, P., Watakulsin, P., Chaiya, S., Kripattanapong, S., Chantian, T., Bloss, E., … Limmathurotsakul, D. (2020). Case-Control Study of Use of Personal Protective Measures and Risk for SARS-CoV 2 Infection, Thailand. Emerging Infectious Diseases, 26(11), 2607–2616. https://doi.org/10.3201/eid2611.203003

• Freedman, D. O., & Wilder-Smith, A. (2020). In-flight transmission of SARS-CoV-2: A review of the attack rates and available data on the efficacy of face masks. Journal of Travel Medicine, 27. https://doi.org/10.1093/jtm/taaa178

• Kim, M. S., Seong, D., Li, H., Chung, S. K., Park, Y., Lee, M., Lee, S. W., Yon, D. K., Kim, J. H., Lee, K. H., Solmi, M., Dragioti, E., Koyanagi, A., Jacob, L., Kronbichler, A., Tizaoui, K., Cargnin, S., Terrazzino, S., Hong, S. H., … Smith, L. (2021). Comparative Efficacy of N95, Surgical, Medical, and Non-Medical Facemasks in Protection of Respiratory Virus Infection: A Living Systematic Review and Network Meta-Analysis (SSRN Scholarly Paper ID 3768550). Social Science Research Network. https://doi.org/10.2139/ssrn.3768550

• Leung, N. H. L., Chu, D. K. W., Shiu, E. Y. C., Chan, K.-H., McDevitt, J. J., Hau, B. J. P., Yen, H.-L., Li, Y., Ip, D. K. M., Peiris, J. S. M., Seto, W.-H., Leung, G. M., Milton, D. K., & Cowling, B. J. (2020). Respiratory virus shedding in exhaled breath and efficacy of face masks. Nature Medicine, 26(5), 676–680. https://doi.org/10.1038/s41591-020-0843-2

• MacIntyre, C. R., Seale, H., Dung, T. C., Hien, N. T., Nga, P. T., Chughtai, A. A., Rahman, B., Dwyer, D. E., & Wang, Q. (2015). A cluster randomised trial of cloth masks compared with medical masks in healthcare workers. BMJ Open, 5(4), e006577. https://doi.org/10.1136/bmjopen-2014-006577

• Matuschek, C., Moll, F., Fangerau, H., Fischer, J. C., Zänker, K., van Griensven, M., Schneider, M., Kindgen-Milles, D., Knoefel, W. T., Lichtenberg, A., Tamaskovics, B., Djiepmo-Njanang, F. J., Budach, W., Corradini, S., Häussinger, D., Feldt, T., Jensen, B., Pelka, R., Orth, K., … Haussmann, J. (2020). Face masks: Benefits and risks during the COVID-19 crisis. European Journal of Medical Research, 25(1), 32. https://doi.org/10.1186/s40001-020-00430-5

• Mitze, T., Kosfeld, R., Rode, J., & Wälde, K. (2020b). Face masks considerably reduce COVID-19 cases in Germany. Proceedings of the National Academy of Sciences, 117(51), 32293–32301. https://doi.org/10.1073/pnas.2015954117

• Offeddu, V., Yung, C. F., Low, M. S. F., & Tam, C. C. (2017). Effectiveness of Masks and Respirators Against Respiratory Infections in Healthcare Workers: A Systematic Review and Meta-Analysis. Clinical Infectious Diseases, 65(11), 1934–1942. https://doi.org/10.1093/cid/cix681

• Payne, D. C., Smith-Jeffcoat, S. E., Nowak, G., Chukwuma, U., Geibe, J. R., Hawkins, R. J., Johnson, J. A., Thornburg, N. J., Schiffer, J., Weiner, Z., Bankamp, B., Bowen, M. D., MacNeil, A., Patel, M. R., Deussing, E., CDC COVID-19 Surge Laboratory Group, & Gillingham, B. L. (2020). SARS-CoV-2 Infections and Serologic Responses from a Sample of U.S. Navy Service Members—USS Theodore Roosevelt, April 2020. MMWR. Morbidity and Mortality Weekly Report, 69(23), 714–721. https://doi.org/10.15585/mmwr.mm6923e4

• Ueki, H., Furusawa, Y., Iwatsuki-Horimoto, K., Imai, M., Kabata, H., Nishimura, H., & Kawaoka, Y. (2020). Effectiveness of Face Masks in Preventing Airborne Transmission of SARS-CoV-2. MSphere, 5(5). https://doi.org/10.1128/mSphere.00637-20

• Van Dyke, M. E., Rogers, T. M., Pevzner, E., Satterwhite, C. L., Shah, H. B., Beckman, W. J., Ahmed, F., Hunt, D. C., & Rule, J. (2020). Trends in County-Level COVID-19 Incidence in Counties With and Without a Mask Mandate—Kansas, June 1–August 23, 2020. MMWR. Morbidity and Mortality Weekly Report, 69(47), 1777–1781. https://doi.org/10.15585/mmwr.mm6947e2

Goodwin has spent much of her adult life studying and writing about US presidents. As a young woman, she worked for Lyndon B. Johnson; and later went on to write a half-dozen presidential biographies. Now in her seventies, she decided to take a step back and have a fresh look at the men she'd studied, this time with one specific focus—leadership. She writes:

Are leaders born or made? Where does ambition come from? How does adversity affect the growth of leadership? Do the times make the leader or does the leader shape the times? [...] Is leadership possible without a purpose larger than personal ambition? How fondly I remember long and heated sessions over just such questions with my graduate school friends, arguing through the night with a fervor surpassing our level of knowledge.

Revisiting these questions five decades later, what she doesn't produce is a dry, academic treatise on the historic principles of leadership. Instead, she tells the story of four presidents, letting their lives and not her words show what made them great leaders. The result is a joint biography of Abraham Lincoln, Theodore Roosevelt, Franklin D. Roosevelt and Lyndon B. Johnson; a biography that doesn't just tell the story of the men and their times, but of great leaders and how they led.

Goodwin structures her book in three sections, with a chapter on each president in each section. The first section is their early life: how and where they spent their childhood, their character development as young men, and the circumstances that first brought them into politics. The second section is all about adversity: how each man experienced a deep crisis in his life and the effect this had on his career, his character, and his leadership. The third section sees them take the White House, facing challenges that threaten an entire nation. “There”, as Goodwin writes, “at their formidable best, when guided by a sense of moral purpose, they were able to channel their ambitions and summon their talents to enlarge the opportunities and lives of others.”

It is no accident that Goodwin chose these four. Aside from the fact that they are the presidents she has studied most thoroughly, Lincoln and the two Roosevelts are known as perhaps the greatest presidents the United States have had. Johnson makes an interesting addition, his legacy and memory scarred by his administration's handling of the Vietnam war. However, he nonetheless deserves great respect for the wide-reaching civil rights legislation he implemented. Also, his somewhat more problematic character (which Goodwin can write about from first-hand experience) makes a thought-provoking narrative counter-point to the almost saintly Lincoln.

For me personally, the book came with perfect timing. I had been wanting to read more about American history for some time, I value good biographies, and I was on the lookout for a fresh book on leadership. Goodwin's presidents covered all bases admirably.

Her writing is delightful to read and she gives one a good feel for the atmosphere of the times she is describing. Switching between the presidents (and the decades) between chapters can be a bit confusing, but also provides an illuminating birds-eye perspective of the broad sweep of American history. She keeps up a good pace throughout her stories, but not at the expense of relevant detail. Her characterisation of her subjects is warm and personal and up-close; making the reader feel as if one has really gotten to know these men, how they thought, felt, and acted. Lastly, her commentary on their leadership never feels imposed or forced. She doesn't use their lives as plump illustrations of her ideas, rather, her comments serve to delicately accentuate the principles they lived.

Quite frankly, I found her book inspiring. But what did it teach me?

Although the book holds many lessons, the three overarching principles it drives home with force are the importance of character, teamwork, and communication.

Indeed, character is what Goodwin focuses on the most. Time and again she emphasises how these presidents combined fierce ambition with a deep-seated desire to truly help others. Surprisingly, they were humble men—not slinkingly 'umble as Uriah Heep, but humble in the sense that they really cared about the person opposite them. Almost paradoxically, it was this humility that drove them on, that made them want to make things better for people around them, and that made people around them trust them enough to give them ever more responsibility and power. They were, in fact, servant leaders.

The greatest example for this (thoroughly biblical) principle is Lincoln. Goodwin remarks of him: “He never allowed his ambition to consume his kindheartedness.” Although he dreamt of greatness from an early age, he was always a compassionate and patient man. A key trait of his personality was that he had a kind eye for those ranked lower than him. As a lawyer, he would talk with even the youngest clerk; as a president, he visited his soldiers in their camps and kept the White House door open for them. He knew that personal was not the same as important. When the Civil War broke out shortly before his inauguration, he assembled a cabinet of the most capable men in the Union—which happened to be his greatest rivals, men from both parties. It took all his determination, empathy, and leadership to hold this group together, but eventually he forged them into a team of deeply devoted colleagues.

This makes him an excellent example of a good teamplayer, too. I found an even more striking example, though, in the story of FDR. After an attack of polio left him crippled in a wheelchair, he assembled a small, intimate group of friends to be his advisers and co-workers. This team (which included his wife Eleanor) became an “extension of his body”, his “eyes and ears” where he couldn't go. From this nucleus, Roosevelt's team grew in proportion to his responsibilities. As Governor of New York, a special advisory committee of three professors brought him into contact with experts in any field he needed to learn about. But his finest hour, and one of the most inspiring stories of teamwork I know, came when he was elected as president.

Defeating the incumbent Herbert Hoover, FDR came to power in 1933, at the height of the Great Depression. America's economy was in free fall: unemployment was at 25%, and thousands of banks had collapsed or were in imminent danger of doing so. As the backbone of the economy, the recovery of the banking system was of immediate and paramount importance. Accordingly, within hours of his inauguration, President Roosevelt set his new-formed cabinet the task of addressing the crisis. Within a day, he closed all banks nation-wide by executive order, imposing a week-long “bank holiday” to halt the tumble and give the administration time to come up with emergency legislation. Now, his team kicked into overdrive. Working around the clock, the president and his ministers consulted with bankers and governors, congressmen and officials to find a way forward. Before the week was out, they had drafted a bill and presented it to a special session of Congress. With both parties working in unison, it passed both the House of Representatives and the Senate in an astonishing, record-breaking nine hours.

The new law provided a framework to begin the long, slow climb out of the Depression. But it could only work if people trusted it, and if they renewed their trust in the banks enough to deposit their money again. Thus, Roosevelt's last act of his crazy first week in office was to speak to the people. In the first of his famous “fireside chats”, he addressed the nation by radio. Simply and personably, he explained “what has been done in the last few days, why it was done, and what the next steps are going to be”. And when the banks reopened the next day, the people responded—across the country, long queues of people waited not to take their money out of the banks, but to put it back in. The tide had turned.

The glowing examples of FDR's fireside chat or Lincoln's Gettysburg Address mark them out as gifted communicators. But although Teddy Roosevelt and LBJ perhaps aren't primarily remembered for their great speeches, they too knew how to connect with people. Each of these four presidents knew the importance of good communication. They could reach out to people and touch them; inform, persuade, and inspire individuals and societies. And this, together with their character and teamwork, made them leaders.

This is what the book showed me. Although I have read many books on leadership (and I heartily recommend reading a more systematic book too, if you're interested in the subject), Goodwin's “Lessons from the Presidents” brings theory to life by simply looking at the lives of these four great leaders. They were four men who made a difference—four men who knew how to leverage their own gifts and those of others, who weren't perfect but who worked hard to make the world just a little better. Because, as was said of FDR: “his general attitude was that the people mattered.”

Why does this bother me, sitting safely on the other side of the Big Pond? Because I can see it affecting my friends, and because the same societal patterns are beginning to manifest themselves here in Germany, too. America was the first modern democracy and we have learnt much from its experience and example. But right now, I find myself praying: “God, don't let us become like the United States.” The Land of the Free and the Brave has become a poisoned democracy.

In Rick Riordan's “Percy Jackson” fantasy series (some of my favourite books, by the way), there is a summer camp for halfbloods. It's the only place where these children, demigods born from the union of a Greek god with a mortal, are are truly safe. To keep out the ever-present monsters, a magic pine tree guards the camp, protecting its borders.

One day, a traitor poisons the tree. As it turns yellow, shedding its needles in a slow death from the inside out, the camp boundary becomes permeable. Monsters start to come through, round-the-clock border patrols must be organised, and of course a quest is dispatched to find a cure for the tree.

In a way, democracy is similar to Camp Halfblood: a safe haven in a world that tends much more towards corruption, oppression, and dictatorships than it does towards freedom and the rights of citizens. And like Camp Halfblood, democracy has a good spirit protecting it: the willingness to solve problems by talking.

A common misconception is that democracy is all about the right to vote. It's not. It's about the right to talk, or more specifically, the right of every citizen to have a say in the attempt to overcome society's challenges. Voting is simply a necessary abstraction for this in a country too large to give everybody a physical seat at the table. Democracy is not about voting for who gets to be the Big Boss, its about discussing what kind of country we want to live in and how we plan to get there. In ancient Athens, a core concept of their democracy was parrhesia, the right of the free man to speak up in the assembly of the city. That is what they meant when they coined the term demokratia, “the rule of the people”.

That, incidentally, is also why the Parliament ought to be more important than the President. A president without parliament is simply an elected dictator. (And believe me, plenty of dictators started out that way.) “Parliament” comes from parler, meaning “to talk”, and that's what parliament is for. This is the institution where the great debates of the age ought to take place, where a group of citizens gets together to talk through the challenges facing our society.

Of course, the debates in parliament are merely an extension of the discourse going on in society at large. And it is a hallmark of a healthy democracy that there is a lively culture of political discussion. The first thing dictators do is to get rid of this, suppressing communal problem-solving and exchanging it for The One True Way (their's, obviously). They know that when the people start talking again, openly discussing the situation, they're in trouble. That's precisely what happened thirty years ago in East Germany. The people started talking, and eventually a wall fell and a despotic government crumbled.

What happens when people stop talking voluntarily?

Now it may seem strange to accuse Americans of not talking enough. After all, few countries have such a strict interpretation of “free speech” as the USA. But the real question is not whether you talk, it's how you talk.

As I said above, the idea of democracy is that we want to solve our problems together. The difficulty is, of course, that you and I may have very different ideas about what the best solution is, or even what the problems are. So we need to talk.

If you've ever sat down with a group of people with diverging opinions and tried to find a common solution (even if it was just about what game to play with your siblings), you'll know how challenging this can be. Some people, after all, are quite attached to their opinions. (I certainly am…) But if you want to find a solution that works for as many people as possible, you need to get off your high horse for a moment and actually listen to what the other person is saying. You have to try to understand him, his situation, and his motives. What is he proposing? Why is this important to him? Could this work for you too, or can you think of an alternative approach that addresses both his and your concerns?

Ideally, you'll come up with a solution that everybody is happy with, or can at least live with. Of course, sometimes you won't. But while it's not such a big deal not to reach a deal on which game to play tonight, there are some issues where a No Deal would be catastrophic. (And no, I'm definitely not looking at you, UK…)

Obviously, the attitude with which you go into the negotiations has a huge impact on their likelihood of success. If you want to play chess and your brother doesn't, he isn't likely to change his mind if you insult him as “too stupid for a thinking game”. The best way to sink a negotiation before it begins is to a) question the other person's motives, or b) insult him personally.

And now we‘re at the crux of the matter. Because almost all Stateside political comments I have read recently neatly fall into one or both of these categories. Somehow, it's not enough to disagree, you have to do so disagreeably. Instead of simply sticking to the facts, you have question your opponents’ intelligence for not coming to the same conclusions as you. Instead of trying to understand what's important to them, you come out with blanket accusations of dishonesty and corruption.

And so you entrench yourself in your cocoon of like-minded partisans, assuring each other that you are right and that the other side is not only wrong, they are evil. You are no longer citizens working towards a common solution, or even parties with differing ideas, you are enemies – and because you're obviously on the side of the Goodies, the others have to be the Baddies.

Thus, democracy is poisoned. With every Facebook post condemning the “Left” and every tweet blasting the “Right”, every article degrading a group of people for holding the wrong opinion, every campaign speech criminalising the opposition. Our German president recently said: “It is through their sum, the daily cannonade of attacks, that many small injuries turn into the gaping wounds afflicting and eroding our culture of discourse today.” It matters how we phrase our disagreement, what words we choose to criticise those of differing opinion; for the words you choose today will affect how you see each other tomorrow. Death comes by a thousand paper cuts. And as the tree withers, the monsters invade.

Because a society that has forgotten how to come to an agreement still needs to make decisions, the feuding parties resort to subterfuge and tactical maneuvering. Instead of a round table of common discussion, politics becomes a battlefield where every achievement for the enemy is necessarily a loss for oneself. Consequently, all means are legitimate if they only serve to frustrate the opposition. Parliamentary procedures meant to expedite the decision-making process are used to prevent a real debate. Constitutional constructs drawn up to manage emergencies are fired indiscriminately until they become daily occurrences.

But democracy was not designed for this. A German constitutional scholar famously said: “Democracy lives on prerequisites it cannot itself guarantee.” Although our constitutions make allowance for dealing with individual misbehaviour, they cannot indefinitely sustain a wholesale attack on the trust underlying the system. Democracy only works when people are willing to talk, and when they trust each other enough to actually do so. And so every attack on the fundamental credibility of a party or an institution ultimately becomes an attack on democracy. Because where people stop talking, they start fighting. Suddenly, democracy has become anarchy.

It's happened before. It happened at the end of the Roman Republic, where an increasingly corrupt Senate took to settling political scores using gangs of thugs on the streets. It happened in the run up to the American Civil War, when tempers became so heated over the question of slavery that war seemed like the only way out. It happened in the Weimar Republic of Germany, when communists and fascists resorted to pitched battles at campaign events, and public trust in the democratic institutions was so far eroded that the people voted the openly anti-democratic Nazis into power. When the tree is poisoned, the monsters invade.

We have to keep talking. It sure isn't easy, but who ever said democracy was supposed to be easy? Talking doesn't mean pretending there are no problems, or that there are no differences in opinion. Talking does mean respecting those differences in opinion and not disqualifying somebody from the debate just because you don't agree. After all, the very concept of a debate presupposes a diversity of opinions. That's something we've just got to live with – it's the price of freedom.

I'm not some big politician or journalist, I don't have the kind of influence it would take to change a culture. I cannot go questing for the Golden Fleece to magically heal the tree. But I can watch what I think and what I say. I can remind myself to see others as fellow citizens with their own ideas about things. I can try to understand where they are coming from – if only to help me persuade them with arguments that are actually relevant to them. And who knows? I might even learn something from them. Perhaps they do have a point. I'll never find out if we don't keep talking.

Optimisation

Thankfully, we are no longer in an era where every byte of memory is precious and programmer time is cheaper than computer cycles. Today, most programmers rarely have to worry about performance issues when writing their code.

Unfortunately, computational scientists often still do. Whether we deal with huge data sets (such as genomic sequences or imaging data) or construct complicated mechanistic models, we are often pushing the boundaries of what our computers can do. So if we find our program's memory consumption exceeds the RAM we have available, or the run-time starts to be measured in weeks instead of minutes, we need to optimise. This takes some skill.

The first rule of optimisation is: do it last! In the words of Donald Knuth: “Premature optimisation is the root of all evil.” First make sure your program is correct, then figure out where the performance bottlenecks are, then try and make them more efficient.

Figuring out performance bottlenecks first, before you start tweaking your program, is incredibly important. Usually, one small part of a software is responsible for the lion's share of its run-time or memory consumption. Improving this bottleneck can get you orders of magnitude more efficiency. And as long as the bottleneck remains, improving all the other pieces is virtually useless.

To figure out where these bottlenecks are, you can use a program known as a “profiler” (every language has at least one available). This will tell you in detail which function calls use how much time and memory. Alternatively, if you just need a quick-and-dirty overview, you can use your language's time() call to calculate function run-times yourself.

(Keep in mind that you can optimise for processing speed and a low memory footprint – but at some point, you are usually going to have to settle for a trade-off between the two.)

Three common types of bottlenecks that you will face are those related to algorithms, data structures, and I/O (input/output) calls.

Algorithms and data structures are a huge component of computer science as a discipline that I won't even try to cover here. Generally, you don't need an in-depth knowledge of the various algorithms anyway, but it does help to at least be aware of tried-and-tested solutions to common problems like sorting or searching. Data structures is something you should try to learn more about, as the right choice of data structure can make a world of difference to your program's performance. You can have a look at this online resource, or check your local library for a relevant text book on the topic.

Whereas algorithmic optimisation is a very mathematical endeavour, I/O optimisation is much more about the hardware you're working on. Often, it isn't the actual calculations that make a program slow, but the time it takes the computer to move the bits and bytes around. Common operations that take a lot of time are network connections, disk reads/writes, or screen output. If you do a lot of these successively, the effect will be noticeable. Here, it often helps to cache data in memory (instead of re-reading it every time it's needed), or buffering output data and flushing it out in one go (instead of setting up a new connection for every little bit).

Overall, optimisation is a huge topic that quickly leads into highly advanced techniques based on very specialised knowledge. Therefore, I will say no more about it, except to encourage you to keep looking and learning, to find out what works in your scenario in your language on your machine.

Licenses

Strictly speaking, licenses are not an aspect of software development, but if you work with or create open source software, you'll have to know at least a bit about them.

“Open source” means that, unlike for most commercial software, the source code for a program may be freely inspected, modified, and redistributed. There's a whole range of licenses that allow this, each with slightly differing conditions. It can all be a bit confusing, but fortunately, there are good overviews available that can be consulted.

Perhaps the two most important are the GNU General Public License (GPL) and the MIT license. The GPL is what is known as a “copy-left” license – not only must the source code for the work itself be made freely available, but all future derivative works must be published under the same license. The MIT license is less restrictive: it only stipulates that any redistribution of the software must retain the original license notice, but derivative works may choose a different license model.

But don't be too worried: if you're choosing a license for your own software, it basically boils down to a matter of preference (and software-political opinion) – unless your employer has a stated policy in the matter. Αnd if you're joining an existing project, that choice has already been made anyway.

Going on from here

Slowly, the importance of good software development practices is being recognised by the scientific community. Recently, the topic was even covered by Nature. Various groups are forming to train, support, and connect scientists who write software – examples being the Software Sustainability Institute or the Research Software Engineer Association.

As I have previously said, the purpose of this series of articles is to give an overview of topics that must be considered when developing software, especially in a scientific context. I hope this in itself proves useful, but must stress that everything I have touched I have touched only very briefly. Much more could and perhaps should be said – but diving deeper remains an exercise for the reader.

In closing, I can only repeat my mantra from the beginning: to become a better programmer, you must read, write, and repeat. In that spirit, happy hacking!

– – –

Read on

1. Principles of Software Development

2. Understandable Software

3. The Art of Abstraction

4. Dealing with Errors

5. Programming Tools: Languages

6. Programming Tools: Paradigms

7. Developing in a Team

8. >> Final Thoughts <<

Bibliography, part II

Apart from the books cited before and the associations mentioned above, here are three more sources for further reading:

• Wilson et al. (2014) “Best Practices for Scientific Computing” – Things to keep in mind when developing code (largely similar to the topics covered in this series).

• Wilson et al. (2016) “Good Enough Practices in Scientific Computing” – Things to keep in mind when working with computers in science generally. Includes a brief section on writing code, but also touches on things like data management and project organisation.

• Netherlands eScience Center guide – A comprehensive guide to software development produced by the Dutch expertise center for research software. Goes into a lot more detail on many topics covered in this series.

Exchanging code: version control

(If you already know about git, you can skip this section.)

The first issue you'll face is how to pass your source files around. Everybody needs an up-to-date version and you mustn't break anything if two people have changed the same file – how do you do that?

For this purpose, there is what is called version control software, the most popular of which is git. In a normal git setup, the code is hosted in a central repository. Each individual developer can check out this code (i.e. download it to his own machine) and change it at will. Each batch of changes is committed – this creates a “snapshot” of the project in the current condition. Finally, the developer pushes his changes back up to the master repository, from were others can pull them into their own local copies. Git merges all changes from all developers so that nothing is accidentally lost or overwritten. (As there are plenty of excellent tutorials out there on how to use git, I won't go into details. See here, here, here, or here for more.)

The most popular git hosting site by far is Github, which you have almost certainly heard of before. It is free and allows you to easily share your code with just about anyone. A popular alternative is Gitlab, which has the advantage that you can host it yourself onsite.

An important point to make is that you should be using git anyway, even if you're just working by yourself. The series of commits creates a great timeline to follow your project's progress, and the ability to jump back to a given commit means you've always got backups in case you screw something up. Also, sharing your work with others is dead easy if you've already got it sitting on Github.

Improving code: collaborative development

Developing software in a team offers enormous benefits, if done right. For a start, the quality of the software itself can increase dramatically, because there are more minds thinking about it and more eyes looking out for mistakes. At the same time, the team benefits too, as its members exchange ideas and share their experience and knowledge. This effect is especially strong for junior team members, who can learn rapidly from observing the way more experienced developers work.

Over the last few decades, industry practice has identified various techniques to maximise these benefits. Two of the most efficient are pair programming and code inspections, which I will briefly outline here. Both of these have been found to have an error-detection rate of around 40-70%, which is even higher than good testing achieves. (Disclaimer: I've never actually had the chance to try these out myself, sadly. I'm mostly going by McConnell's Code Complete 2 and various online sources such as this.)

Pair programming

Pair programming is a bit of an unusual approach in that it entails two programmers working on the same computer. One, the driver, is the one actually typing into the keyboard. The other, the navigator, watches for mistakes and thinks ahead about what must be done next. The two partners keep up a conversation on what they are doing or thinking and regularly switch roles.

This technique is especially effective when working on difficult sections of the code. Pairs keep each other focussed, spot more mistakes and think of more possible approaches than an individual developer would. Together, they deliver higher quality code in a shorter time than alone.

However, it takes a bit of practice to get used to this style of coding. Not all pairs combine well, and not all code is amenable to the technique's specific benefits. When it works, though, it works well; and developers often enjoy the team-oriented approach.

Code inspections

Code inspections are a highly formalised version of code reviews that are intended to find errors during the development phase. An inspection will be chaired by a designated moderator (not the author of the inspected code!). This team member sends out a copy of the code to a small number of reviewers (about one to four) and arranges a meeting for the actual discussion. Each reviewer prepares by reading the code himself and annotating any problems he finds, based on a checklist of common mistakes/good practice. During the meeting, the code is read through section by section as the reviewers give and discuss their comments. Importantly, the aim of the meeting is not to find solutions, but purely to identify problems. Each problem is recorded and the complete list is later handed to the author for fixing.

Inspections are a very thorough tool that has proved to be highly effective at finding errors. As McConnell writes:

A study of large programs found that each hour spent on inspections avoided an average of 33 hours of maintenance work and that inspections were up to 20 times more efficient than testing (Russell 1991).

Although they might seem overdone in their formality, experience has shown that less structured code reviews usually perform significantly worse. One challenge in the inspection is to keep the discussion on point and technical. This is where the moderator comes in, as it is his task to ensure that the discussion doesn't drift off-topic or become personal (attacking the author).

– – –

1st Postscript. The NASA space shuttle software is renowned as being the most bug-free software ever created. Through stringent development practices, its engineering team managed to reach a rate of just one error in half a million lines of code. (Industry average is 1-25 errors in 1000 LOC!) For an instructive write-up of how they do this, see here.

2nd Postscript. I have created a simple code inspection checklist for use at our institute. If you are interested, you can download the basic version, or a self-explanatory version.

Read on

1. Principles of Software Development

2. Understandable Software

3. The Art of Abstraction

4. Dealing with Errors

5. Programming Tools: Languages

6. Programming Tools: Paradigms

7. >> Developing in a Team <<

8. Final Thoughts

I have to say I'm rather sad about having to switch. I really liked Mendeley. It had a great user interface, an inbuilt PDF reader and annotator, easy syncing, good web and mobile apps, and useful Word/Firefox plugins for citing/importing. In short, it was just what I needed. (Plus, it was free, and it worked on Linux.)

I first got a bit dubious when the company behind Mendeley was bought by Elsevier. My supervisor is vehemently anti-Elsevier, and I confess I have no great love for their business practices either. But practicality beats purity, and Mendeley was great – so I stayed.

Disaster struck some time last year, when I opened my library one day as usual, only to find I couldn't open most of my PDFs anymore. Like a ball in a magician's hand, they had simply disappeared! Soon I discovered that I wasn't the only user affected, and that the development team had somehow managed to push a buggy update that had mangled files all over the world.

Thankfully, I had just handed in my bachelor's thesis, or I would have been completely screwed. In my then-current situation, it was merely an inconvenience that I hoped would soon be righted. But not only did Mendeley take several weeks (!) to release a fix, it so happens that the fix didn't work on my machine. Even now, most of the entries in my library are orphaned.

As I am now back in a stage where I have to do a lot of literature research, that basically spelled the end of my Mendeley time. I can't work with a broken library, and I won't fix a library if I can't trust the hosting company not to break it again (or even to care that it's broken). So I knew I finally had to make the switch to Zotero.

As I read up on making the switch, I discovered two other things about Mendeley. First, their Word and Firefox plugins are basically forks of older (and inferior) versions of the same Zotero plugins. Secondly, a couple of months ago, they started to encrypt their library databases. With the result that Zotero can no longer import Mendeley libraries, even though the reverse is perfectly possible. In other words: Mendeley not only broke my library, they are now preventing me from salvaging it with a working program.

It turns out I still had an old backup lying around that was still unencrypted, so I have been able to recover about three quarters of my library entries for Zotero. But as you can imagine, I was still pretty upset.

So now I'm on Zotero. My general impression is favourable so far, although some things are still suboptimal. My biggest lament is the loss of the Mendeley PDF editor. For some inexplicable reason, there still isn't a decent general PDF editor available for Linux, at least not on a GNOME desktop environment. Meaning, I can no longer directly annotate my PDF files. Oh well.

Slightly bothersome was the fact that several features that I was used to from Mendeley are only available via plugins in Zotero – but at least they are available. (For example, creating a constantly updated BibTeX file from a collection requires the “Better BibTeX” plugin.) I'm also going to miss the Mendeley mobile app, although I understand Zotero has hired an iOS developer to build one. (Hope it comes out soon!)

Other things, however, are significantly better. I am especially pleased with the import process. Zotero is a lot better at identifying article metadata from the PDF file. On Mendeley autorecognition worked about half the time, on Zotero it's more like 90% – that saves a lot of effort! Also, the web importer (Firefox plugin) actually works, and is usually spot on.

I like the coloured tag system, and the ability to attach HTML notes to items. Most of all, I am relieved to know that this is open source software that I can fork myself, if necessary; and that I can sync my library using my own WebDAV server – without having to trust any external company.

To sum up: I have traded in a bit of convenience and some very useful features in the switch away from Mendeley. But given that I can no longer trust the company, I can live with that. And in several ways, Zotero really is better.

Briefly, a programming paradigm is a way of thinking about programming. Specifically, it describes how we go about representing a real-world problem in computer code. A paradigm therefore includes both a mindset of how to analyse these problems, and a process for implementing the solution.

As computer science and programming practice advance, paradigms wax and wane in popularity. Here, I will briefly outline the three that are currently the most frequently used.

Procedural Programming

This could perhaps be considered the “default” paradigm. It is what is known as an imperative paradigm, meaning that programs are thought of as a series of commands for the computer. Programming, in the procedural paradigm, consists of step-by-step instructions to the computer about how to solve a given problem. (“First do this, next do that.”¹)

On the implementation side of things, code is structured in blocks: loops, conditionals, and functions. (Hence the name of its historical predecessor, “structured programming”.) Generally, functions are viewed not in their mathematical sense, but merely as a collection of instructions that are grouped together for convenience reasons. In short, they are not so much functions as “procedures”.

If you write a simple script, chances are you're using this procedural paradigm. Its approach is also very prominent in the next paradigm, object-orientation.

Object Oriented Programming

If you're taught software development today, OOP is what you're taught. This paradigm was developed in the eighties, caught on in the nineties and was standard for the development of larger programs by the 2000s.

Its core idea is to view the world as a collection of interacting objects of various types. To do this, it uses classes, objects, and methods.

For example, imagine you wanted to model a city. A city consists of houses, so you would have a class House. Houses can do things, like consumeElectricity() – this would be a method. But not all houses are alike: there are appartment blocks, schools, factories, offices… Each of these subclasses inherit from the base class House, meaning they can all do that a basic house can (such as consume electricity). But they can each have more specialised methods, too. For example, the class School may have the method teachChildren(). (In the context of OOP, a method is defined as a function that can only be applied to values of a defined type; in other words, a function that is associated with a class.)

But there's not just one school in a city, neither is there just one appartment block. Classes, therefore, are instantiated to produce objects. An object is a specific instance of a class – not just any school, but “Loughby Secondary School”. Each object has all the methods and attributes defined by its class and any superclasses, but has its own attribute values. (All schools have a name, but every school has a different name.)

This is a very intuitive approach to many programming problems, especially those that involve modelling real-world phenomena. It also encourages strong encapsulation and abstraction. Because of these strengths, it is currently the most popular paradigm for anything bigger than a 300-line script.

It does have its downsides, though. For starters, large object-oriented systems have a propensity to turn into a complicated tangle of classes and hierarchies – veritable phylogenies that are about as easy to understand as that of the prokaryotes. More significantly, not every programming problem lends itself to this way of thinking.

Functional Programming

Data processing, for example. Although one can use OOP to represent a data pipeline (such as that from DNA sequencing data over alignment to genome analysis), it is more natural to think about it as a set of functions applied in sequence.

That is just what functional programming does. In FP, the real world is not abstracted into a bunch of classes and objects, but into series of interlocking functions, each representing one process. Functions here are viewed in their mathematical sense, as entities that take some input and provide some output. Another way of putting it is that FP is about “evaluating an expression and using the resulting value for something”¹.

This means that functional code uses a lot less assignment operations than the imperative paradigms (it is considered a declarative paradigm), as expressions can easily be nested. Less assignments mean less variables being used, which avoids bugs caused by the improper/unexpected modification of variable values. Indeed, “pure” functional programming dictates that functions should be side-effect free, meaning they must not modify any global state at all. Given identical input, a function should always return identical output – regardless of the conditions elsewhere in the program.

The major advantage of this is that such functions are almost trivially easy to test and debug, and can thereafter be considered “black boxes”. This greatly aids reliability and understandability. On the downside, deeply nested function calls can be troublesome to read, and completely pure FP is hard to achieve.

A second aspect of FP is that because functions are so important, they are made “first-class objects”. That means that the programming language treats them like any other variable type: you can pass functions as parameters to other functions, or have functions that return functions. This opens up a whole world of possibilities. Amongst others, it enables the map function, which takes in a function and applies it to each element of a given list (a quick alternative to a for loop).

Historically, FP has been around a lot longer than OOP. (Once again, Lisp was a pioneer in this department.) During the height of the “OOP-craze” it very much diminished in importance, but is now being rediscovered as a sensible solution for many scenarios.

Conclusion

Of course, functional programming isn't the Holy Grail either (even if some of its proponents are at least as zealous as many of those of OOP). Remember, the key message here is to Use the Right Tools.

There are many programming problems for which object-orientation is the most obvious solution, and there are many others for which a functional approach is a lot better. Equally, there are some programs that are so simple that you don't need either.

Use whichever is appropriate, and feel free to mix-and-match. Fortunately, most languages today are multi-paradigm and support more than one possibility. (Though, it has to be stressed, not all do equally well in all paradigms!) Just remember: not everything is a nail.

[1] Kurt Nørmark

Read on

1. Principles of Software Development

2. Understandable Software

3. The Art of Abstraction

4. Dealing with Errors

5. Programming Tools: Languages

6. >> Programming Tools: Paradigms <<

7. Developing in a Team

8. Final Thoughts