The scene lives!

Opening a dataset of underground multimedia art

⛶  Open fullscreen

title: The Scene lives! tags: echtzeit description: Echtzeit x GLAMhack 2021 slideOptions:

theme: dark

TL;DR - Presenting (see intro) Echtzeit = Contributors to international demoscene - GLAMhack / OpenGLAM = Community of international culture-data wranglers - Our #GLAMhack 2021 project: - Prototype web app: - Source code app: - Prototype API: - Data + API: - Slack channel: #team-26-echtzeit-x-openglam

10'000 m goals

  1. Make the demoscene more accessible to people who may have never heard about it. (:wave: hello #GLAMhack!)
  2. Create an open dataset of demoscene productions, which could be filtered to individual themes or platforms.
  3. Support the UNESCO digital heritage application, or just explore the history of the 'scene.

..Or just tune into SceneSat :headphones: and enjoy electronic art at the hackathon!

Elevated by Urszula "Urssa" Kocol

Elevated by Urszula "Urssa" Kocol

Our GLAMhack roadmap

:point_down: The drilldown

Road by PG and R0ger

Road by PG and R0ger

Pouet logo by tomaes

Pouet logo by tomaes


The oldest and most well known central repository, pouët, makes daily data exports available at with a JSON API endpoint at and the open source code of it at

We downloaded and tried to parse the raw JSON with a couple of tools, and didn't manage to get far. Convoluted structure and formatting errors were rather demotivating. Nevertheless it influenced our thoughts about a "demoscene data standard", and brainstorm ideas of improving overall data quality (for example, we immediately noticed mismatched dates and missing values).

This API was also recently used to do some terrific data analysis, and we reached out to the authors to find out if we can reuse their scripts. We have also reached out via the #pouet Discord for guru meditation.


The Demozoo API is basic but usable: As it's a paginated web service, it would require a bit of scraping code to aggregate. So we used the 'nuclear option' of getting the database dump in raw SQL format. Importing this into a local SQLite database (inspired by all of Wikipedia in SQLite) and then re-exported the tables in CSV format. This should be done differently for automated data updates.

We have reached out via the #demozoo Discord and GitHub for some further ideas.

Demos Take Control by Der Piipo and Kustaa and Mazor


A popular current way to crowdsource open data in a distributed way is the Data Package, the preferred format for doing this using the Frictionless Data project, which has a create tool to generate an initial datapackage.json.

An initial data package based on the Demozoo archive is at GitHub, which compiles and aggregates the data from several tables using the Python dataflows library.

Toolbox by Frictionless Data


Exploring and transforming the data gives us a frame of reference based on which to think about some commonalities and differences between different archives' approaches. We did some research to see what effort in this direction was already made, and reached out to the Demozoo and Pouet communities.

Each of the data sources have a schema of their own, and some attempts at consolidation have been made. We started with a simplified version of the Demozoo model, created a Table Schema which can be used for validation or annotation as JSON Schema.

Ascii Attack by Latex


Our dataset repository clearly explains its sources, but also points out there are many other places which could be future data acquisition targets. These notable scene repositories and data sources include:

  • (as above)
  • (as above)

Dump V1.71 by Dead Hackers Society


While the Data Package is nice to look at in it's JSON glory, most people (and programmers) will want some kind of interface to it. We wrote a small server using the Falcon Framework to produce a barebones API. Since the data is loaded using a Frictionless Data wrapper for the Pandas library, it can incorporate various advanced sorting and filtering routines. Our proto-service is currently running at a private VPS hosted on Linode, but should also work on 'lambda function' hosts like Vercel or Heroku.

Dream by Vantage

Still from Traffic Jam by Chainsaw


If we make it this far, we would like to make a basic example of data usage. After all this data wrangling, we didn't have time to really explore the space of user interface possibilities. But we have a small application that demonstrates the API with an infinite-scrolling user interface showing productions.

See for yourself at

Ideas to build upon

  • Establish live feed graph view of demoscene productions
  • Create an infographic that helps to explain the demoscene
  • An interactive app for scrolling through prods (e.g. Netflix or Giphy clone)
  • A cheatsheet to learn the most important terms and famous groups/prods
  • A virtual reality exhibition (like other teams are working on)

fr-036: Zeitmaschine

More or less irrelevant links

  • Thread showing various graphs of prod release stats from Pouët
  • Discussion of "greets graph"
  • Some debate about classification
  • Discussion of Demozoo API
  • Oleg's blog explaining the demoscene, partially on the topic of "letting the data speak for itself"
  • Teaser Revision 2017 seminar "Graph databases and the demoscene universe"
  • Ideas from demosceners on GitHub
  • Internet Archive gallery "To some, the heart of the Demoscene - the self-playing examples of programming and artistic prowess of the last 30 years on the underpowered but extremely flexible C64."

Demo or :skullandcrossbones: die!

Dig Your Own Hole by Scape / Mist

:sheep: Thanks for watching!

:love_letter: :bird: @seismist

Pass it forward:

This presentation is shared under CC BY 4.0

This content is a preview from an external site.

Create an open dataset of demoscene productions, which could be filtered to individual countries, themes, or platforms, and help make the demoscene more accessible to people who may have never heard about it. This dataset could be of interest from an art-history perspective to complement our UNESCO digital heritage application - or just be used to introduce people to the history of the 'scene.


Supporting the demoscene, one Data Package at a time
Data Package 🌐 json
  • productions

Prior art

This project is closely related to the Swiss Video Games Directory from previous OpenGLAM events, and was quite inspired by this tweet:

Repository updated

30.04.2021 13:40 ~ loleg

Event finished

17.04.2021 17:30

Edited content

17.04.2021 12:12 ~ loleg

Joined the team

17.04.2021 09:27 ~ kudrix

Edited content

16.04.2021 08:44 ~ loleg

Repository updated

16.04.2021 08:42 ~ loleg

Joined the team

16.04.2021 08:42 ~ loleg

Challenge posted

16.04.2021 08:42 ~ loleg

Event started

16.04.2021 08:30
Contributed 2 years ago by loleg for GLAMhack 2021

Connect to our community on: | twitter | facebook

All attendees, sponsors, partners, volunteers and staff at our hackathon are required to agree with the Hack Code of Conduct. Organisers will enforce this code throughout the event. We expect cooperation from all participants to ensure a safe environment for everybody. For more details on how the event is run, see the Guidelines on our wiki.

Creative Commons LicenceThe contents of this website, unless otherwise stated, are licensed under a Creative Commons Attribution 4.0 International License.

GLAMhack 2021