Project proposal

For the remainder of the semester I would like to focus on learning the d3.js library for data visualization purposes. As a designer I'm often interested in conveying information through visual elements, and that often includes making raw data (or a message, or even just a feeling) accessible and clear to the viewers. Learning programming has allowed me to both turn my works' viewers into participants, or users, and also to create visuals that change over time and respond to real datapoints.

I have a couple of raw project ideas in mind, but I would like to just start exploring different ways of visualizing all kinds of data. I would probably use some APIs or open sources for the data, but ideally this exploration will also include learning methods of scraping data by myself from the web. The main motivation I have for doing this is around the possibility of discovering interesting connections or tendencies that arise from the data, while also visualizing it in a compelling way.

‍

⤷ Ideas for a project title?

Datapoints? Something better will probably come up while working on the project.

‍

⤷ Project description (max 100 words)

Learning the d3 JavaScript library through weekly projects and experiments to visualize various information. The data for the projects might include open sources, APIs, and ideally also data that was sourced specifically for the project, either by manually creating a database, or by learning data scraping techniques. The goals of the project are to explore representation of data in a way that might yield interesting outcomes, as well as to allow me to learn a tool for creating compelling data-driven visuals.

‍

⤷ Who is the user? (the intended audience can also be you)

I think the audience for this type of work would be people that find interest in this type of thing — designers, developers, technologists, and maybe even statisticians and the like.

‍

⤷ What is the user experience?

For the most part I assume it would be things like buttons and sliders to filter through information, but I definitely plan to look into more novel ways of interacting with the data and exploring interactive possibilities within the data-vis space. I think it'll ultimately depend on the goal of each specific project, so I wouldn't force it, but I would definitely do whatever works best concept-wise.

‍

⤷ What is the creative aesthetic/angle/approach?

It'll depend on each specific project, but it is a very two-sided thing for me — I'm sure some of the time I'll start by deciding what data I want to work with and then figure out the form, but other times I could look at examples and ask "what kind of information would be interesting to display in this form?" and start from there.

‍

⤷ What technologies are required? Why?

I'm not sure if and how p5 might come into play, as p5 is canvas based and d3 is vector (svg) based(?) so using both might be unnecessarily overcomlicated. I'll start with some d3 examples, and I assume those will include using some databases, and reading and writing JSON and CSV files. Google Spreadsheets might come in handy too for basic data analysis. If I do manage to dive into some data scraping techniques I assume some use of Python will be needed too. As I have no previous experience with that I'll have to see if it makes sense in terms of the scope.

‍

⤷ What do you need to learn/research/decide?

I will need to learn all the mentioned above — d3.js, working with databases, JSON, CSV and SVG files, and methods of scraping data if possible.

I did read a little about data scraping and relevant methods that came up were: HTML parsing, DOM parsing, and using the IMPORTXML function in Google Spreadsheets. I obviously don't want to get into anything that might be problematic (legally), I just see the potential in maybe working with datasets that don't already exist out there so I'll need to figure out what's possible.

‍

⤷ What assets/content/code already exists? (your own sketches or other people's sketches)

The web is full of d3.js examples which will surely be a good starting point for some explorations. Last semester I tried using Google Spreadsheet as a server with a very simple concept, but it was laggy and I ended up not doing anything with it. Since then I've learned how to set up an actual server, though I guess this is a quick way of prototyping. I have also come across a p5/d3 cookbook that features a number of examples that seem like a good resource.

As for the data — there are many sites that link to many more datasets. There's NYC Open Data, and data.gov that include lots of interesting datasets, from NYPD arrest data, to squirrel sights in central park. There's this list of datasets on Github that includes things like human genomes, cloud analysis, international trade statistics, household electricity study, data on many politicians, a lot around covid-19, a lot of image analysis databases and many many more. Unfortunately it looks like a lot of the links on this list are no longer live, but I assume I could search for the datasets I'm interested in by name.

‍

⤷ What assets/content/code needs to be created?

I think it's mostly just finding or assembling interesting datasets to work with.

‍

⤷ What resources/books/sites/examples will you rely on?

I have linked to a few above. I'm sure there's many more, but I'll try to start from something simple and progress gradually from there. This d3 introduction seems good, as well as this one. This one is very simplistic but was good as an easy entry point (I couldn't wait so I started with that).

I also came across the book Visualizing Data by Ben Fry. It looks like mostly Processing / Java and not d3, but it still seems like a good source for understanding how to deal with large amounts of data and for doing good data analysis and data visualization in a more general sense.

‍

⤷ What is the end goal?

Two main goals: One is to have a number of working and hopefully interesting data-vis projects. The second goal is to become acquainted with d3.js and to learn its capabilities.

‍

⤷ How will you determine if this project is a success?

"...There's no win and no fail. There's only make. The only rule is work. If you work it will lead to something..." — Corita Kent: Ten Rules for Students and Teachers

‍

⤷ Cite 3 sources of inspiration (How did you become interested in this idea? Quotes, photographs, products, projects, people, music, political events, social ills, etc.)

1. I have always liked seeing data visualizations, be it charts or maps or weirdly shaped graphs. I think it's extremely powerful in understanding complex behaviors and in making them accessible to others. I have also always liked creating visual work. I have recently found myself interested in working with data, either in this class (sorting algorithms, webcam pixel data) or in others (using markov chains, getting data from the browser, using APIs like Whois to access public data, and even getting and visualizing solar energy from the sun). Yes, using data is a very broad theme, but I guess I just like not having full control, and setting a perimeter and letting the data do its own thing.
2. Last semester I took a class called "spreadsheets for game designers". In one of the classes the professor gave us a quick assignment in groups of 3-4 students. We were all given a dataset of a sports team that shows how many points each player scored in every game for a year. Each student group had to analyze the data to make their case as to why their (assigned by the professor) player was the best player. When everyone finished presenting their case, someone asked the professor "well, so who is the best player?" and the professor just said "I don't know. The data can make it seem as if any of them is".
3. I went to a data-vis conference last year and found a lot of the work presented there really impressive and inspiring. I don't think any or most of the presenters worked with d3 but a lot of them had a playful and detail-oriented approach I'd like to channel myself.

‍

⤷ Do you think all of this feasible in 7 weeks? Why?

I realize "Data-vis" could be a whole class (or a whole Masters even) but I think I should be able to familiarize myself with a javascript library in 7 weeks(?). I feel comfortable with HTML & CSS, and have a little bit relevant experience with Javascript and other related fields like mentioned above. I think a big part (half? I don't know) of the challenge for me personally will be the visual / aesthetic aspect of it, as well as the conceptual aspect, which I find to be very motivating!

‍

⤷ Questions for your classmates. What are you unsure of, conceptually and technically? On what aspect(s) would you like feedback?

Does this sound like too much? Should I make it more focused? Maybe focus on one specific dataset? (that could be very interesting but might also be too limiting). Or maybe choose a theme? (I have a couple of ideas but I don't really feel comfortable committing to anything before I start exploring the options). Any suggestions?