Sampling Distribution & Central Limit Theorem
Understanding sampling distributions is really important, but very few researchers…
Changing pace today, statfans. I tweeted about my winning workflow for academic writing involving Scrivener, Zotero and Pandoc, and a few people asked for a detailed explanation – so here it is.
Credit for this whole setup goes to Dave Smith at Nottingham University. Dave’s how-to is perfectly clear if you’re a very tech savvy Scrivener user. If you’re a comfortable bash-scripter then go ahead and use Dave’s concise tutorial. The instructions below are aimed at everyone else.
Note: This tutorial is for Mac users only. If you’re a Microsoft user, I’m sorry to hear that, and wish you the best of luck on your Scrievner/Zotero journey 😉
Note: Within scrivener, you need to write in markdown for this to work. What does this mean? Instead of using scrivener to put in all of the rich-text formatting you want (italics, bold, underlined, headings, etc) you need to use markdown syntax to embed the formatting into plain text. You can still write using any font you like, but unless you use markdown to express your formatting needs you will end up with an unformatted mess at the other end. Do not panic, this is not a big deal. Mostly, you need to remember to put *one asterisk around italic text* and **two asterisks around bold text**. And specify your headings properly. Click here to learn the basics.
An added bonus of doing academic writing in markdown is that you can write tables in markdown, and don’t have to deal with Scrivener’s table functionality. This is almost worth doing even without the awesome reference integration I’m about to demonstrate.
First, we need to get Zotero to produce a unique identifier for all of the items in your Zotero library. The reference also needs to be in a format which other programs can understand. Without an exportable unique reference for each library item there is no hope of encoding the references in Scrivener. The consensus from people who know more about this than me is that the best way to do this is via the Better BibTeX Zotero plugin.
To install the plugin,
The two steps below are absolutely crucial, double check you’ve got this next part right or nothing is going to work properly. You want Zotero to pass all of its quick-copy citations through the BetterBibTeX plugin, and you want BetterBibTeX to output citation keys in a format which Pandoc can understand.
To set this up, once BetterBibTeX installs, click “preferences” in the extension window and change the Quick Copy Format to “Pandoc”.
Now click “Export” at the top of this window (or navigate via Zotero > Preferences) and change Zotero’s default Quick Copy format to “Better BibTeX citation Key Quick Copy”.
Now click the Better BibTex icon at the top of the preferences window and double check you’ve got “Pandoc” selected as the quick copy format.
Now we need to export your whole Zotero library, into a format that Pandoc can understand. In Zotero, Navigate through File > Export Library…
Select the Better BibLatex format, and tick the keep-updated box. Save your library as something easy like My_Library.bib, somewhere you can easily find it (we’re going to move it later).
This is where it gets difficult if you’re not a tech-y person. Hang in there!
Zotero is now ready to talk to Scrivener, and we have an updated library ready for Pandoc to use once we install it. In this step we’ll set up a simple method for grabbing the BetterBibTex citekeys from within Scrivener.
Dave Smith has written an application which makes this whole process incredibly easy. In your Mac’s Applications folder is a “Utilities” folder. Go there and open Script Editor (or search for Script Editor using the Spotlight Search)
Copy the 50 lines of code from here and paste them into the script editor. Save the script as something like “zotpick-pandoc_Scrivener”, and select “Application” from the File Format drop-down menu before you save. Visit this very clear tutorial if you are confused.
Now we need to point scrivener in the direction of the application. We will do so by setting it as our reference manager. Navigate through Scrivener > Preferences > Citations. Click “choose” and select the application you just saved a moment ago:
Once you’ve done this, when writing in Scrivener you can use the keyboard shortcut (⌘ – Y) to pull up a fabulous little red box which will grab citekeys for you! Type the name of an author or keyword to search your library. Once you’ve selected the reference(s) you want, hit return and the applet will type the citekey within Scrivener, wherever your cursor is placed. This works in any word-processing program or text editor. Visit this page if you need help with the applet.
Now we need to setup Pandoc, which will be able to convert these weird citekeys to proper references.
Pandoc is an amazing program which can convert between a huge number of text file formats. It can also scan text files and replace BibTeX citekeys with correctly formatted references, which is why we’re going to use it. Pandoc does not have a user interface, you use it via the command line, by writing computer code and not by clicking a mouse. Do not panic, Dave Smith has already written all the code you need. Thanks Dave!
Download and install the appropriate version of Pandoc for your machine (probably the MacOS.pkg one) current version at time of writing is here.
If you search for “Pandoc” using the Finder or Spotlight Search, you won’t find it on your machine. This is odd, because you just installed it! Pandoc lives in a hidden directory, and before we can ask Pandoc to scan our scrivener files and populate the references, we need to get into that hidden directory and put some things in.
We need to add:
Open a finder window and use ⌘+ shift + H to navigate to your home directory. Now use ⌘ + shift + . to reveal the hidden directories, and open .pandoc
Move My_Library.bib (from earlier) into this directory.
If like me you want to ultimately export into MS word because your co-authors use it, create a new word document and call it reference.docx. The contents of this document do not matter, Pandoc will use the Styles specified for this document to format all of your output. To change the defaults, open reference.docx in word, click “Styles Pane” and modify any or all of the following styles recommended by the Pandoc Manual:
[paragraph] Normal, Body Text, First Paragraph, Compact, Title, Subtitle, Author, Date, Abstract, Bibliography, Heading 1, Heading 2, Heading 3, Heading 4, Heading 5, Heading 6, Heading 7, Heading 8, Heading 9, Block Text, Footnote Text, Definition Term, Definition, Caption, Table Caption, Image Caption, Figure, Captioned Figure, TOC Heading; [character] Default Paragraph Font, Body Text Char, Verbatim Char, Footnote Reference, Hyperlink; [table] Table.
Once you’ve set all these styles in the way you want, save reference.docx and close Word.
Ok, let’s bake this turkey! Now all this hard work is going to pay off. I’ve prepared some text below, go ahead and put it into a new scrivener document:
Abstract ============ Here is the abstract! The text above is a top-level header. Introduction ---------------- Here is the introduction! The text above is a second-level header. Here is some text which requires a reference. Here is some text which requires multiple references. Here is some *italics*. Here is some **bold text**. Pandoc will populate the reference list at the end of the document, so here is a heading for that. *References* ----------------
Within Scrivener, use the keyboard shortcut (⌘ – Y) to pull up the applet, and add some references from your library to this test document (if you lose the red box just hit F3 to find it). I’ve highlighted my citekeys in yellow, below:
Now export this document as a plain text file (.txt). Don’t put any spaces in the file name because you’ll need to point to it through the command line in the next step, call it “pandoc_test.txt” or something similar.
Now open a terminal window (Applications > Utilities > Terminal)
The code you need to run (thanks again to Dave Smith) is below. Copy the code below and paste it into a text editor like textEdit:
pandoc –bibliography \
–csl ~/.pandoc/vancouver.csl \
-f markdown -t docx \
-o ~/Documents/Folder1/Folder2/Output.docx ~
Now edit all of the coloured bits. Type the name of the relevant csl style which corresponds to the kind of reference you want (note: you need to have copied the csl file in the hidden .pandoc folder for this to work!). Next adjust the file path so it points to the folder in which you’d like the final word document to appear. And finally, point pandoc to your text file you want it to add the references to.
Once you think your code is ready, paste it all into the terminal and hit ‘Return’. Pandoc should think for a second, and if no error is returned your word document is ready! Close your terminal and go take a look! The result of my test is below. Here is the code I used:
And the final result is here. Notice that the heading style specified in reference.docx has over-ruled the formatting indicated by the asterisks in the “References” heading.
If you want to go even further, you can preview your fully-formatted document while you write using Brett Terpstra’s Marked 2. I haven’t explored this yet, but intend to in the future.
I hope you enjoy this workflow. And, if you don’t work with code often, I hope you enjoy feeling like a programming ninja whenever you run your pandoc script 🙂