Building the Computer Literacy Project archive

The BBC Computer Literacy Project ran throughout the eighties, giving us the brilliant BBC Micro and inspiring a generation of programmers - including me. When the first programme of the project, Managing the Micro was transmitted in March 1980 I'd already spent many weekends coding on an Apple II Europlus borrowed from my dad's work and then enjoyed an entire summer holiday (during which I didn't see much sun) with a Commodore Pet 4016 kindly loaned from school by my maths teacher. I was already hooked - and incredibly excited that the BBC would make actual TV programmes about my personal obsession.

When word of the BBC Micro's advanced specification and sophisticated version of Basic reached my circle of computing friends we knew this was the machine to have. I'd used its predecessor, the Acorn Atom, and come to love features such as the ability to write 6502 assembly language inline in Basic programs. My much-pestered parents bought me a Beeb Model A for Christmas 1982. Within a couple of months I was spending most of my weekends at the shop where they'd bought it - HCCS in Gateshead - by then a meeting place for like minded folk. Pretty soon I was serving customers and using my pay to upgrade my Model A bit by bit to Model B specification.

The BBC Micro was my first proper computer and the lessons I learnt programming it, understanding its design, expanding and, eventually, modifying it have been the foundation of a life spent joyfully working and playing with computers.

So when, in April 2016, Jake Berger asked if Sam Urquhart and I would be interested in developing a web-based archive of the Computer Literacy Project programmes we were thrilled and honoured.

The archive owes its existence to two veterans of the Computer Literacy Project: Series Producer David Allen and Programme Engineer Steve Lowry. When Sam and I were introduced to them Steve had already made an epic Excel spreadsheet containing all 146 programmes of the original project and scores of related programmes, all broken down into annotated, categorised clips.

Steve also found and rescued hundreds of discs containing the BBC Micro software that was used for on-screen demos (quite a lot of which he'd written in the first place) and obtained digital copies of all the programmes from the original series. He combined all these elements with a custom search interface (written in Visual Basic) that allowed clips to be searched for and played and relevant BBC Micro programmes to be run in an emulator. It's easily the most impressive spreadsheet based application I've ever seen and it presented us with quite a challenge to do it justice with the web site.

Steve's curation of this content was ongoing so his spreadsheet would need to continue to be the authoritative version of the project. Clearly we'd need to be able to re-import the spreadsheet every time additions and changes were made. That gave us an obvious starting point; I wrote an importer that would parse the spreadsheet and populate a MariaDB database with entities representing brands, series, programmes, clips etc. The importer would also use ffmpeg to encode any new media items for HLS streaming and extract a poster frame for each clip in every programme.

I made a rough and ready first draft of the web interface so we could start to get a sense of the shape of the archive and figure out how best to present it. I confess, I was quite happy with that first version of the site. That's when Sam Urquhart got involved in the project. She hadn't seen the original programmes and knew little of the project's history. This allowed her to represent all those people who, like her, would need a gentler introduction to the archive. Frankly, she said my site was awful, opaque and unfriendly. She got to work redesigning it.

I had already seen Matt Godbolt's amazing browser based BBC Micro emulator, jsbeeb, and hoped to use it to make the software that Steve had recovered from all those floppy disks run in the browser. While Sam worked on the front end I integrated jsbeeb. I even got to write some BBC Basic - jsbeeb is launched with a small bootstrap program to display the instructions. I was pleased and surprised in 2017 to be building a web service that generated a program written in a language I hadn't worked with in 30 odd years.

Sam and I had previously worked together to build BBC Genome  and we used some of the same Open Source technology for this site. The back end is written in Perl using the Dancer framework with Template::Toolkit. The database is MariaDB  and the full text search is provided by Sphinx. The interface is built on Bootstrap 3 and the video / audio player is VideoJS. The site is hosted on servers running Ubuntu.

Throughout the development of the archive we have been impressed that so many of the concerns raised then (coding for everyone, women in technology, privacy, artificial intelligence, loss of jobs to technology) have proved to be prescient. As technology continues to transform society these issues are now, more than ever, at the forefront of our relationship with machines.

In common with many programmers of my generation I owe the BBC a huge debt of gratitude for taking such a bold initiative - the benefits of which are still with us today. It was a great privilege for us to work on this archive. We hope you enjoy it.

You can access the Computer Literacy Project archive at https://computer-literacy-project.pilots.bbcconnectedstudio.co.uk/