As uTheory has grown, so has the computer code that powers it. Roughly 185,000 lines of code (LOC) power uTheory's server, webpage, app and administrative tooling. For context, many programs have millions of lines of code. But our codebase is now larger than what any one programmer can fully hold in their head.
To let us move faster at building music learning tools, and not break things, we've spent much of 2021 improving our codebase with more reliable technologies and practices. This post is quite technical, but we wanted to share with you a bit of what we're up to!
So, we've moved the vast majority of our codebase (85%, or 154k LOC) to TypeScript, a language that allows you to specify what kind of data any variable can hold. Typescript provides tools so that whenever you use that variable, you're given hints to prevent doing illogical things (like trying to divide "Hi, Greg!" by 32).
Automatic Unit, Integration and Regression Testing
Alongside of this, we've vastly increased the percentage of uTheory which is automatically tested, focusing on those parts of the codebase which are used most widely. Before we deploy a new version of uTheory, these tests check to be sure things work as expected. Given the musical nature of our codebase, the testing process can be delightfully cacophonous:
But it helps us make sure that by changing line 65, we didn't accidentally cause line 143,201 to break!
Upcoming Data Migration
And even early on we had problems with its performance, and turned to work-arounds, like storing data for class grade-sheets separately from the original student data, which added undesirable complexity to our server code. In the years since, we've simply outgrown RethinkDB.
Over the past month, we've completely rewritten uTheory to use PostgreSQL, a battle-tested relational database with strong data type guarantees. We've written and tested code to move existing data to our new Postgres database, and on Sat., January 29th at 10pm Eastern, we'll take uTheory offline for somewhere between six to twelve hours to make the transition to using Postgres.
On the surface, not much will change. At first, all of uTheory's functionality will remain the same. But, under the hood, uTheory will be operating on a chain of technology that is designed to help programmers move quickly in large codebases.
That's because with Postgres we know what the shape of our data is. In combination with Prisma, it enables our developers to rely on autocomplete rather than their own memories. Check out how little typing I actually do when writing a function to load a user's skill progress. The autocomplete (or "intellisense") practically does it all!
In the coming months we'll be rolling out new features and expanding configurability for uTheory. From new rhythm and ear training exercises and lessons, to more robust progress tracking, we think you'll love what our new tech stack is going to make possible!