The write-up below isn't exactly the same as the one found in the Data Sketches book. For the book we've tightened up our wording, added more explanations, extra images, separated out lessons and more.
I just want to say how absolutely relieved that I'm now finishing my last month of Data Sketches. For a little while I was feeling a little bit of despair that maybe I'll just never be able to finish this last project. Indeed, I spent two years trying to create a data visualization with Taylor Swift's work - first with the colors in her music videos, and then with the emotions in her lyrics (detected by IBM Watson's Tone Analyzer). I learned a lot from the process, and I really liked the visuals that I had come up with, but I couldn't figure out what I wanted to tell with the story. I was stuck in limbo for a year.
At one point, I had to admit to myself that I might never be able to finish Data Sketches if I kept on having writer's block. So I started thinking through other projects I was already working on, that could be a compelling last project for Data Sketches. And I realized that the final project I was working on at ITP was as good of a candidate as any.
For the last few years I've been wanting to bring my data visualizations into the physical world. I've been obsessed with the idea of expressing data-driven stories as installations that people can actually walk through, to explore and touch and feel. Once I realized that the biggest reason why I couldn't design any projects in the physical world was because I work in the 2D, and have no idea how to think in the third dimension. As soon as I realized that, I created Legends with Three.js and WebGL. That project really helped me learn how to think about space, lighting, and camera position.
Bug being able to code in 3D isn't enough to create a physical installation, I also needed to learn hardware. In the fall of 2019, I did an Artist-in-Residency at New York University's Interactive Telecommunications Program (ITP), a master's program that is at the intersection of art and technology. I went to ITP with a big hope: to create a physical data installation in collaboration with talented people.
I audited Introduction to Fabrication where I learned about power tools and machines and played with the laser cutter, and Introduction to Physical Computation where I learned about circuits and sensors and lights and Arduinos. And through those classes (and the office hours I was asked to hold as a resident) I was able to meet my two project partners: Christina Dacanay with her beautiful illustrations and bold uses of color, and Tina Rungsawang with her architectural background and keen awareness of space. I knew they would be perfect complements to my data and code background.
When I asked them if they would do a final project with me, they immediately agreed (and I was so happy). We decided to aim for an installation around women in computing to be exhibit at ITP's Winter Show.
From the very beginning, we agreed that we should center the project around women in computing. It's a topic that I've been very interested in since I did Legends with women Nobel Laureates, and Tina and Christina were in complete agreement that it'd be a very fitting pursuit at ITP (which combines code and computation with art).
I started my data gathering the same way as Legends, with a Wikipedia article. I was able to find a page titled Women in Computing, and from there I discovered the Wikipedia category Women computer scientists. In this first attempt, I wrote a script that requested the page, used JSDOM to turn the HTML text into JSON, and iterating through that for the women's names (I've done this many times for previous Data Sketches projects also, whenever I had to scrape a webpage). I then used wikijs to get more detailed information from each woman's Wikipedia pages, including number of backlinks, references, and links in the page, as well as their basic information and summary.
Once I had the initial dataset of 416 women (it was really cool to see so many women computer scientists listed), I wanted to see if there was something interesting for us to center the installation around. But I had to first filter out any women whose pages were stubs and didn't have basic information. For the 247 women who remained, I took a look at their distribution of birth years, and number of backlinks and references. Nothing was out of the ordinary, so I decided to concentrate on the women with more than 100 backlinks that may have more fleshed out Wikipedia pages (I thought backlinks might be a good proxy for completeness).
With those 29 women (which included Tamara Munzner!), I took a look at the intersection of their backlinks, links in the article, nationalities, alma maters, common words in their summary...the only things that stood out were that majority of their nationalities, when listed, were American (which only showed that this list is unfortunately skewed towards America). I was hoping that clusters would appear, like if some were known for their HCI work, or for their backend optimizations work. But those patterns didn't emerge, most likely because of the simplicity of the data I decided to pull (just their summaries instead of their whole articles) and because of the simplicity of my own analysis (I was merely counting intersection of keywords in summaries).
Instead of trying to devise more sophisticated analysis (we were quite limited on time), I decided to take a step back and try my hand at another dataset. I had initially avoided scraping the Women in Computing article since the women's names were scattered throughout the paragraphs instead of in a list, and it was much less straightforward to scrape. But once I realized that the data was much richer there - it succintly described each woman's accomplishments and even gave an approximate timeframe for most women's contributions - I knew it'd be worth the trouble.
For this second attempt, I wrote a script to:
Once I had the spreadsheet, I went through each person to double check their gender. For each of the remaining 46 women, I wrote a one sentence summary and found an approximate year for their contributions by reading through the Women in Computing page and cross-referencing their own Wikipedia pages. I also in the process highlighted approximately 30 women that I found particularly inspiring and brilliant.
Because of the space constraint for the Winter Show, we could only design an installation with a maximum of 20 women. For the final pass, Christina, Tina, and I all sat down and narrowed the list down to the 16 women that inspired all three of us.
This is probably the most different and most interesting sketch section from all the other Data Sketches projects I've ever done. It should more accurately be called "prototype".
For our very first brainstorm, we asked ourselves: what would we want to make if we had no restrictions on time, space, and money? We went through many different ideas, but at the core, we agreed that we wanted to highlight the invisibility of these women, and raise awareness for their incredible contributions in computing. We wanted to create an experience that would have people leave feeling inspired, stimulated, and wanting to go through it again. We also wanted to create something where people's interactions with the piece can alter it, and they could maybe even leave something of themselves behind.
The last point of participants being able to affect the artwork was the most important to me, as I've always loved work like Domestic Data Streamers' Golden Age Art Installation where people are invited to add to the artwork and change how it looks at the end. I love that it turns people into participants, that it gives them a sense of ownership in the artwork.
For the second brainstorm, we brought in installations we pinned for inspiration, and a lot of the aesthetic we seemed to converge on were installations with many small objects hung and suspended in mid-air. We especially liked Judith G Levy's Memory Cloud, where nostalgic photographs are placed within small souvenir viewers. People are invited to walk through dozens of these viewers, and peek in to see each 35mm slide.
We loved the idea of people interacting with each object, and decided to map each object to a woman and put their information inside. We wanted people to take their time reading about each woman, and I really liked that looking into the viewer felt like looking into a secret world that we previously had no idea about.
From there, the rest of the data mappings came quickly: the women would be organized in a field by the year of accomplishment, from earliest to most recent. And their height from the ceiling would be by the number of their backlinks. So the more backlinks and more "renowned" they were, the higher up they were because they were more "out of reach", and the less backlinks the lower down they were to be more easily accessible. This was especially important to us, to promote those women that were less well known. We wanted the interaction to be that a person would pull the object down from above their heads and read inside the object. We thought that interaction could also be how we keep track of which women were interacted with, how we give user feedback, and how we potentially affect the end artwork.
To have people be able to leave something behind, we decided to have a "summary board" placed at the end of the installation with each woman's name on it. We wanted to prompt people to place a star sticker (hehe) by a name that they had never heard of before, but were inspired by after the installation.
(We initially wanted our experience to have a clear entry and exit, with the summary board placed by the exit.)
Because this installation was also the final project for the Physical Computation class I was auditing, there were regular check-ins on the idea and scheduled "playtests". In our first idea check-in, our teacher gave us the feedback that it'd be really nice that walking up to and pulling down that first object would also trigger other objects to drop down. That feedback inspired us to cluster the women into groups, so that when a person walks up to the first woman, the others in the group would drop down and create a path to guide them through the installation.
Once we had the initial idea fleshed out, Christina went about sourcing materials, and Tina and I went about trying to figure out how to detect a person interacting with our object. We brainstormed several ideas, including a pulley system that would complete the circuit when pulled down on (thus sending a signal), a motor at the top that would unwind the cord when pulled on, a capacitive touch sensor on the object, an infrared motion sensor placed somewhere near the object...all the options felt overly engineered, overly finicky, or overly sensitive.
When we went to our Fabrication teacher's office hours, he suggested we go simpler. He gave us a vibration sensor (the circuit completes when we shake the sensor), and a tilt sensor (the circuit completes when it's tilted up) to experiment with.
As you might be able to tell from the gif, the vibration sensor took quite a bit of force to trigger, which meant that if we put it in our object the person would have to really shake it (instead of gently picking it up) for us to know that they've interacted with it. On the other hand, though the tilt sensor wasn't perfect (there didn't seem to be an exact angle that it started sending "on" signals, and even then those signals were quite noisy) it was a much better option.
(We also considered an accelerometer, which would be more sophisticated than a tilt sensor and be able to tell us the angle of the tilt, but we ultimately decided against it because it was ~4x more expensive than the tilt sensor.)
Through this process, we also realized how finnicky a motor would be to deal with and fragile it would be especially with the thousands of people coming through the Winter Show and pulling on the cords (with many kids!). Even though we really liked the idea - motion always really captures attention - we decided to nix the idea. Instead, we decided to put lights in the object since changes in light are still quite noticeable and even more importantly, much easier to control.
We immediately liked the metaphor the lights gave us: we decided to have the hanging objects all start out dim - a metaphor that these women are largely unknown - and as people went through the installation and read about each woman, their light would grow slight bit brighter. We hoped that by the end of the show, all of their lights would be brightly lit.
For the lights, we went with Adafruit's brand of individually controllable RGB color lights called the Neopixel. We originally went with this because we thought we might try encoding data into the color of the lights as well but decided against it because it was quite hard to distinguish most colors from each other unless they were very different (like between, red, blue, green). We decided to continue using it even though we weren't going to have different colors, because they gave us better control over brightness than normal LEDs.
Unfortunately, when we first ordered the Neopixels, we ordered 100 individual ones, not realizing we'd have to solder each of them to their own boards so that we could wire them up to the arduino to communicate with. I learned a very valuable lesson that first time I soldered one of these Neopixels, that they have four pins (one for plus, one for minus/ground, one for data in to receive commands from the Arduino, and another for data out to send the commands to the next pixel if we ever decided to chain them) very very close together and they CANNOT touch. And if they touched, they would short-circuit.
(Trying to hold down the wires so they wouldn't move as I very carefully tried to solder the pins to a cut-up board.)
I did four more of this. Unfortunately, two of them shortcircuited and we decided it really wasn't going to scale up very well. On the bright side, I eventually learned how to use a multimeter to test for shorts, instead of trying it out on my (and Tina's) Arduino boards and shorting them...
Fortunately, because there are very many talented and experienced people always on the ITP floor, they were around to advise us on using a Neopixel strip instead, since it has a much nicer and larger area we could solder to. Unfortunately, those larger areas were still quite close to each other and if we weren't being careful, ran the risk of the solders running into each other. Amidst those struggles, I was introduced to soldering paste, which makes it so that the solder only sticks to the conductive part and doesn't stick to anything else. That was such a time saver. And after soldering the wires to the strip, I learned to use a hot glue gun to make sure that the wires wouldn't fall off easily.
(These sorts of lessons are precisely what I went to ITP for 👍)
At the same time we were figuring out the electrical components for each object, Christina was sourcing and experimenting with materials. She quickly landed on using fillable Christmas ornaments, and we went about acquiring one from a nearby Michael's to prototype with and ordering three dozen from Amazon at the same time.
Christina then went about sanding the nobs off of each ornament, and carefully drilling holes into the center for wires to run out of. To hide the tilt sensor and Neopixel within the orb, we first tried to spray paint the interior of the orbs. We had bought reflective spray paint that was supposed to give a mirror-like finish, but it didn't turn out well. That was the night we learned reflective spray paint only works well on glass and not plastic 😅
We also took the opportunity to test out two ideas we had: whether we wanted people to peek in through a small hole to read the information, or whether we should leave the orb as is and have a larger area to read the information from. It was really great to have the spray painted prototypes, because right away we could tell that the peephole would be way too small and too much of a struggle for people to read large amounts of information through.
(The two options we considered for how the orb should look like)
We also realized that we should place the information placard at a certain angle so that people would 1. know that there was a placard inside they needed to read, and 2. be incentivized to pick it up in the correct direction to trigger the tilt sensor.
(Prototype of the second option with no peephole, and a large transparent area to read the information through)
It was really fun to prototype the materials, and the whole process was so different from working digitally (where gathering materials is not a thing). Overall it took about two weeks for us to settle on all the materials, and we had to be really strategic about buying just enough material to test with without being wasteful if we decided against the idea, and ordering the materials online early enough that it'd get to us by the time we needed to start production. I also learned a lot about prototyping with cheap, readily available materials (like masking tape, cardboard, and tin foil) to mimic the effect we were going for just to be able to quickly try out an idea.
Since the reflective spray paint didn't work out, Christina suggested cellophane (Christina was an absolute pro when it came to materials). This not only hid the tilt sensor and neopixel, it also had a beautiful effect when lit up in the dark.
(A quick prototype with cellophane and the information printed on vellum, a translucent paper to help mimic frosted acrylic and diffuse the light, that can also be printed on.)
Once we had all the physical components for the orb determined, I went about with the coding. To control the tilt sensor and Neopixel we settled on, we used the Arduino.
Arduino is a brand of microcontrollers (I think of microcontrollers as small, simple computers that we can use to control other devices), and the way we use it is by wiring the pins on the Arduino (circled in orange in the photo) to our lights, sensors, motors, etc. Each of those pins have a corresponding number it is identified by, and so in our code, we can tell Arduino that a certain pin is being used as input to read from (usually a sensor) and another is being used as output to write to and control.
(You never thought you were gonna get a run-down of Arduino from this project, did you? Well we aim to go above and beyond 😆)
Because this was my first full project with Arduino, I started my code super simple. I used a button as a stand-in for the tilt sensor, and an LED in place of the Neopixel. I used an array to keep track of which pins the buttons were wired to, and another array for which pins the LEDs were wired to. I also had an array for keeping track of which buttons were pressed, and another array for each LED's brightness. Every time the button was pressed it was the equivalent of a "tilt" and I'd increase the brightness of the LED and save it to the brightness array. I wanted to use the button and LED (which I was already familiar with how to code) first so that I could get the functionality working first, before having to worry about incorporating the tilt sensor and Neopixel - both things I haven't worked with before.
(Four Neopixel strips and one orb with Neopixel and tilt sensor all frighteningly wired up for my code testing purposes.)
Once I had the basic functionality, I went about replacing the LED and push button with the Neopixel and tilt sensor in my code. The Neopixel actually comes with its own library of functions, and because it assumes that I'm programming a strip of many Neopixels, most documentation examples involve instantiating the Neopixel class with an array of pin numbers. I actually had to do some gymnastics and guesswork to setup each orb's Neopixel individually. (Next time, I might consider using something else, but for this project it was already too late to look for other options.)
Reading the tilt sensor was a little more straightforward, though I did have to adapt an Adafruit example to implement debounce on the reading. That was probably the most interesting lesson I learned about working in hardware, that nothing in the physical world is clear cut like it is in computers, and all sensor readings are noisy.
With the Neopixel and tilt sensor code incorporated, the rest of the code from my first iteration worked perfectly.
(I got this working around 1AM after a few late nights and I was so deliriously happy.)
With one orb working, I had to make sure it could work scaled up and with groups. We had decided to group our list of 16 women into four categories - Mathematicians, Computer Scientists, Creatives, and Others (executives, inventors, etc.) - and if any one orb was lifted up, the rest of the women in the same group would also light up in a delayed stagger.
(Working out the logic for when an orb is picked up and all the other orbs in the same group should light up in a staggered pattern, and the opposite happening when the orb is put down.)
To implement the delayed stagger, I needed to incorporate time into my code. The only way to do this in Arduino is with mills(), which returns the number of milliseconds since the Arduino began running. I refactored my code to store group information: whether the whole group should be lit up or not, which orb in the group triggered that change, and at what millisecond that change was triggered (which I called startTime). I could then use that information to loop through each orb in the group, and depending on which orb triggered the change (activeOrb), calculate each orb's startTime (when its own brightness would change) based on the distance of that orb from the activeOrb. I would then set the orb's brightness if the time was past that orb's startTime.
(The Neopixels lighting up when the tilt sensor in my hand is lifted, and dimming down when the tilt sensor is down.)
I also ended up coding in a fade animation by storing a start and end brightness for each orb, which made the brightness change much gentler, and added in code for a pulsating introductory orb to catch attention and explain the installation.
Coding in Arduino was an interesting challenge in trying to make sure I was being memory efficient, especially when I had to add in groups and remember milliseconds (which were stored as unsigned longs). It definitely made my code more convoluted, but I had a really fun time trying to figure out the challenge.
This was the point at which I had finished my fellowship and had to go home to SF (and then to a conference in Tokyo), so I only was able to help Tina and Christina finish prototyping the orbs and wasn't able to help them through the production process. I highly recommend checking both of their writeups (Christina's and Tina's coming soon) for more details on this section.
One of the first things that Tina did to kickstart the production phase is to 3D model an enclosure to hold both the tilt sensor (which is very sensitive to the angle of its placement) and Neopixel in place. We also received a brilliant suggestion from our teachers to use a phone jack to connect our orbs to our wires, so that we could make the orbs easily detachable. That way, just in case an orb malfunctioned, we could easily separate it and debug it away from the rest of the installation. Tina also incorporated the phone jack into her enclosure design, and together with Christina, 3D printed 17 of these enclosures and spent roughly 30 hours soldering them all.
(3D model of the enclosure.)
(Tilt sensor and Neopixel soldered to the phone jack within Tina's 3D printed enclosure.)
(The orb wired up and working as a stand-alone piece!)
They also designed the information within each orb, and laser etched and cut it into acrylic.
And finally, they translated the data I gave them into positions and heights in the space we were given for the Winter Show. They then used planks of wood to hang the orbs from the ceiling.
(Install timelapse)
It was absolutely incredible to see this whole project come together. It received great feedback at the Winter Show, where over two hundred people experienced it. My favorite stories are of women, young and old, reading through each orb and looking inspired and emotional, placing a gem in the groups they most identified with.
(A board where we asked people what groups they identified as - definitely not the most optimal Venn diagram, but probably much easier to understand for non-dataviz people.)
I'm so grateful to Tina and Christina for being the best teammates to pull off this ambitious project (in 6 weeks!), and to all the teachers and residents and students who graciously offered their time and wisdom. I don't think we could have pulled this project off anywhere else.
And I'm so happy and grateful that this is the project that I get to end a whole decade with, to end Data Sketches with. I think it perfectly encapsulates my personal journey - of going from someone that was purely technical, to learning more about design, to finding and chasing and executing on my dreams of doing a full data installation. I'm excited that this ushers me into the next phase of my career, where I want to do more art, and bring more data visualizations out into the physical world for more people to get immersed in and enjoy.
Thank you for three years, Data Sketches 💖
For the last few years, I've been wanting to bring my data visualizations into the physical world, to express my data-driven stories as immersive installations that people can walk through, explore, touch and feel. I struggled with this dream for years, until I realized why I felt such a mental block: because I designed and coded primarily in 2D and not 3D, I had no idea how to even think in the third dimension. As soon as I realized that, I created Legends with Three.js and WebGL, and it really helped me understand how to think about space and lighting in the third dimension.
But being able to code in 3D isn't, by itself, enough to create a physical installation; I also needed to learn hardware. In the fall of 2019, I did an Artist-in-Residency at New York University's Interactive Telecommunications Program (ITP), a master's program that is at the intersection of art and technology. I went to ITP with two major hopes: to learn more about hardware, and to collaborate with talented people on a physical data installation.
To learn hardware, I audited Introduction to Fabrication, where I learned about power tools and machines and played with the laser cutter, and Introduction to Physical Computation, where I learned about circuits and sensors and lights and Arduinos. Through those classes (and the office hours I was asked to hold as a resident), I was able to meet my two collaborators: Christina Dacanay with her beautiful illustration style and bold uses of color, and Tina Rungsawang with her architectural background and keen awareness of space. As soon as I saw their portfolios, I knew they would complement my data and code background perfectly.
When I asked them if they wanted to collaborate with me on the final project for our Physical Computation class, they immediately agreed. We decided to create an installation featuring women in computing to exhibit at ITP's Winter Show.
