Research Update 4: Organizing, Storing, and Otherwise Making Data Retrievable

To be honest, I generally have no idea what I’m doing. I think this is, in fact, the usual feeling of the average graduate student. My tendency, when I don’t know what to do, is to organize. While sometimes this results in bouts of procrastination-induced spring cleaning, abroad I don’t have these types of distraction since I’ve only got the one room and almost nothing to organize. So, I make to-do lists, five-year plans, and data spreadsheets. I also offer to do all the dishes.

Since organizing is my go-to method of dealing with the unknown, before I began my research year abroad, I approached the library staff at UCSD about my options for managing and storing the vast amount of data I was sure to be collecting over the course of not only the next year, but probably the next decade or so. I met with a bunch of different people, which felt kind of weird, but also awesome, and we figured out what we think will be the best platform for me. Below, you can find the questions I asked, or was asked, to determine my organization and storage needs, as well as the solutions we developed in response to those questions.

Things to think about when choosing data management and storage solutions:

  • What do you want from your data management system? I want to be able to easily access all of my data in one place, and to be able to search by key words or themes, all in order to make the writing process as easy as possible.
  • What will your storage needs look like? I knew I would have a lot of images. Turns out, from my time in Nantes alone, I have over 12,000 images. In addition to space, I need to be able to easily retrieve my data as well as to make sure it is all saved and backed up, many times over.
  • What kind of data are you working with? Pretty much all of my data are written documents. For the most part, I’ve been able to take pictures of the documents. So, I’m mostly working with images I’ve taken, though I also have descriptive items for those times when I wasn’t allowed to take pictures.
  • How will you use your data in the long-run? I plan on using the data I collect this year for at least the next decade, probably (nothing happens quickly in academia). I want to be able to access, organize, and search the data easily and quickly in order to facilitate writing and analysis.

The result is a multi-pronged approach that gives me endless storage space, the ability to organize my data (which includes images, metadata, and commentary and analysis) in one place, and easy access. So, what does this look like?

  • Data entry. As I briefly outlined in my post on my time in Nantes, I had a daily routine that involved downloading new images and inputting new data. For inputting data, I have an Excel spreadsheet (it’s a template that’s formatted for the storage and data platform I’m using but follows the one that I made for my work at the Arab American National Museum, which you can see in this post) that allows me to include all the metadata (image number, archive and collection name, box and folder numbers and names, image descriptions, commentary/analysis, and key terms/themes) for each image. Unsurprisingly, this is the most time consuming of my tasks and I am, in fact, still making my way through writing close descriptions of each of my images from Nantes, three months later.
  • Storage. My images and spreadsheets are stored in three places: 1) two 1-terrabyte external hard drives, 2) Google Photos (if you don’t need archive-quality images, you can upload an infinite number of photos here; my spreadsheets are saved to Google Drive, rather than to Google Photos, for obvious reasons), and 3) my data entry + storage platform, discussed below. I’ve organized my images the same way on both my drives and on Google Photos to make access easier.
  • Data entry + storage + access. The final piece to the data entry, storage, and access puzzle is a platform called Shared Shelf. We decided to go with this because it’s made for storing images and the metadata that goes with it, and can be customized to include the things that I want like analysis and thematic tagging for each image. By including these tags, accessing my materials by theme can be accomplished by selecting the desired tags. My initial commentary and analysis will be helpful for reminding me why I might have taken the picture, a particularly important aspect of the document, or a helpful translation. I’m only able to get this through my library’s subscription, and it will remain available to me once I’ve graduated.

So that’s that. A (kind of) quick overview of how I organize all the material I’m collecting abroad so that it stays safe and can be easily accessed and manipulated once I get to writing.

*  *  *

What kind of data organization systems do you use, whether it’s in daily life, at work in a non-academic setting, or in the academic world? What kinds of tips and tricks do you have for making the process of data management and processing a little less cumbersome?

Research Update 3: Centre des archives diplomatiques de Nantes

Part I of my research year took me to Nantes, France (see my previous post for more general, non-research related thoughts on my time there). I worked for six weeks in the Centre des archives diplomatiques in Nantes (CADN), a branch of France’s national archives that deals with documents produced by the diplomatic office and holds materials for the French mandate in Syria and Lebanon between World War I and World War II.

Below is a rough daily schedule of my time in Nantes to give you an idea of what my six weeks working in the archive looked like:

Some time between 6:00 and 7:00 AM – Wake up and contemplate the day: Should I go for a jog first? What’s the temperature like? Do I want to have to shower after? (The usual conclusion) Naw, I can get some stuff done and enjoy a slow morning if I don’t.

6:00 – 9:00 AM – Eat breakfast, enjoy a leisurely coffee/tea, check Facebook (my main connection to home) and news, download pictures from previous day and update Excel spreadsheet if not done the night before.

9:00 – 9:30 AM – Head out the door to the archive. I lived a little ways away, so I had a 10-minute or so walk to the tram, 15-minute tram ride, and 15-minute walk to the archive. It was generally a nice time to catch some brisk fall air, a walk, and to organize my thoughts for the day ahead.

9:45 – 10:15 AM – Arrive at the archive and have awkward conversation with security using my limited French. Be nervous about the metal detector for no reason.

10:15 AM – Choose my seat and request the first box of materials, simultaneously hoping for a hidden treasure and for there to be nothing helpful; the former obviously necessary for furthering my research, the latter to reduce time spent taking and processing pictures.

10:15 AM – 1:00 PM – Work through boxes requested the previous day. In the first couple weeks, I’d go through about 3 boxes in the morning before lunch and 4 after. As time went on, the boxes I requested had fewer relevant materials (being no longer from the Public Instruction archive) and I sometimes would finish with all 7 (the daily limit) by 1:00.

1:00 – 1:30 PM – Request boxes for the next day and eat lunch (this I brought with me from home; it always included a baguette. So good). This was also an opportunity to sometimes chat with other researchers, which was always pleasant.

1:30 – some time in the afternoon – How much time I spent at the archive for the rest of the day was largely dependent on whether or not I found relevant materials. I noticed that  people had all kinds of techniques for working through the documents. I tried two main techniques: 1) Going quite methodically through materials, taking notes on relevant documents and taking pictures, filling out my spreadsheet as I went 2) Going methodically, but just taking pictures of relevant documents while at the archive and making my notes when I got home. The second technique seemed to work better for me, as it required less time in the archive, which helped me avoid the dreaded “archive fever” (related to cabin fever and referring to feelings of restlessness and boredom resulting from extended presence in an archive) and allowed me to process my work in the comfort of my own place with food and a tea, coffee, or glass of wine to aid me.

Afternoon – evening – This was generally my processing time (though this sometimes was moved to the morning, depending on how tired I was from the day’s work–translating and picture-taking are surprisingly mentally exhausting–or if there was a particularly good French crime drama or guardian angel show on). Processing entailed downloading the day’s pictures to my hard drive, backing them up to my external, and then uploading them to Google Photos. I’d also update my Excel spreadsheet with the relevant data (what that looks like to be shared later).

Obviously, very exciting stuff. Stay tuned for more research updates on data organization and management, daily schedules, my time doing research in Beirut, and my trip to the League of Nations archive in Geneva. For now, I’d love to hear in the comments about what others’ archival work days look like, how you avoid archive fever, etc.