Why Putting JSON in a Database Column is a Bad Idea. Or, Why Would You Ever Put JSON in a Database Column?

When I was a beginner at databases, I was tempted to put JSON in a string column. It’s a weirdly common thing for beginners to relational databases to want to do. A lot of tutorials aimed at beginners suggest doing it for some reason. When people show up on Stack Overflow wanting to know why their code isn’t working, and that code happens to be putting JSON in a relational database column, sometimes someone will show up and tell them to quit doing that, but it’s only like a 50/50 shot, whereas pretty much every other questionable beginner practice will attract hordes of smartasses, some of them barely more than beginners themselves, who will strongly and sometimes stridently censure the beginner for contemplating it. I attribute the popularity of Mongo and other NoSQL databases in part to this instinct: Mongo is basically a database with nothing but string columns that have JSON in them, and some search functionality for JSON in those string columns.

If you’re sometimes tempted to put JSON in a string column, I hope I can explain today why it’s a bad idea and you shouldn’t do it, in a way that will make sense. On the other hand, maybe you’re one of those relational database savants who understood third normal form right away, and you don’t understand why anyone would ever want to put JSON in a string column instead of having a real schema. If so, I hope I can explain to you why someone might want to put JSON in a string column.

Strong Static Typing, Relational Data Modeling, and Escape Hatches

In “Is Weak Typing Strong Enough?”, Steve Yegge describes (buried somewhere in the point he was actually making about programming languages) how teams that he worked with at Amazon were too boxed in by the strong statically typed strict schema the relational data models imposed on them, and resorted to tactics like an untyped name / value system and passing an XML parameter as a string to a CORBA interface. (CORBA was a standard for an early style of distributed services that let you treat objects running on remote servers as if they were in the same address space and could call each others’ methods. It was also language agnostic, so you could run Java on one server and APL on another server and they could use their CORBA implementations to call each others methods. At least, that’s what Wikipedia said; CORBA predates my programming experience by some years.)

The point here is that relational database models are a sort of strong static typing. No matter how much you love strong static typing, sometimes it’s not flexible enough, and you need escape hatches. In 2005 when Steve Yegge wrote his piece, it was XML strings through CORBA. On a programming language level, it’s things like downcasting from Object in Java or using the dynamic keyword in C#. Storing JSON in a string column is another one of these escape hatches. What you get is flexibility. Let’s keep this in mind as we go into our big example, which I hope will demonstrate both why you shouldn’t put JSON in string columns and why you might want to sometimes.

Hugs and Warm Fuzzies: An Example

This example is loosely based on a real system I worked on that had JSON in string columns.

Let’s say we’re working on an MMORPG called Crayon Art Online. Unlike most MMORPGs, which are all about killing things, Crayon Art Online is all about love and friendship and positivity.

Every few hours, one of our servers needs to kick off a job that will read a bunch of player actions from the game’s main database and calculate metrics on them, storing them in a separate metrics database. Since the game is all about doing friendly, happy things, the actions will be things like hugs_given, crayon_drawings_gifted, lunches_bought_for_others, pep_talks, and positive_affirmations. There will be some actions that have extra associated metrics, like hug_warmth and affirmation_positivity and pep_talk_peppiness. There will be some conditions where we just check for a Boolean answer, like sculpting_class_taken?, which just has a true or false answer. We need to calculate all these metrics for all the players across different periods of time: the past 30 days, the past 90 days, the past year, and the entire period the player has existed. Then we need to store them in a database so another job that runs later can read them and distribute prizes to the players that reach certain goals, and messages of encouragement and hope to the others.

Let’s go into some aspects of our data that we’ll have to model. We have a ton of different actions–let’s say there are about 25 total. It’s not a fixed set; there will be new metrics if the developers add new actions to the game, and they might also remove actions, so we would lose the metrics associated with those. That means making a separate table for each metric is doable, but a huge pain. Every time we add a new metric, we have to add a new table. Reading all the metrics for a player means querying 25 tables. A simple manual check for data consistency between the main game’s database and ours requires us to write a select command that includes 25 tables. It seems there must be an easier way.

However, one table also clearly won’t work unless we store all the values as strings. We need a value column to be able to store integers (hugs_given), floats (average_positivity), Booleans (is_wearing_floppy_hat?), and strings (floppy_hat_color). So another option would be one table per value type: int_valued_metrics, real_valued_metrics, boolean_valued_metrics, and string_valued_metrics. But that’s pretty tacky. Your data model is supposed to model the data, like it says in the name. The data itself doesn’t naturally divide into these different types; they’re all just metrics, with the same fundamental features: a name, a time period, a user, and a value. Our database engine needing to store the value as an integer or float or whatever is an implementation detail, not an actual part of the domain, and it feels unnatural to spread the data model for the single metric concept across four tables (and counting, if you add types later—maybe at some point you have tuple-valued metrics, or BLOB-valued). This is better than 25 tables, and it’s better than one table, but it’s still not good.

At this point, things look pretty dire. Relational data modeling has failed us. But wait, you realize. Every programming language nowadays, well except for Java anyway, has a powerful JSON parsing engine built in that can turn a string of JSON into a native map, with type conversions and everything! So, why don’t we just store metrics as a string of JSON per user?

Strings of JSON have lots of attractive qualities. They’re easy to understand; JSON is dead simple, by construction, whereas SQL and relational data models are complicated and hard to understand. Strings of JSON are easy to parse into whatever kind of map or hash thingy your language supports (unless you’re using Java or Scala), and once you’ve done that, you know exactly what you can do with them. Reading from a relational database usually requires some annoying library that binds the data to native data types, or else you can make raw SQL calls, get handed back the most useless, primitive thing your language supports, and do the conversion yourself. JSON is dynamically typed, so you don’t have to worry about finding the correct data type; just store the string, and when you read it back the parser will figure out what types everything should be. JSON is extremely flexible, letting you store anything from a simple glob of key-value pairs to a bramble of arbitrarily nested lists and maps, whereas with relational data models you’re stuck with two dimensional tables.

So, let’s just forget all this relational modeling stuff and make a table that maps a user id to a string of JSON that looks like this:

{
    “hugs_given”: {“30_days”: 5, “90_days”: 10, “1_year”: 85, “all_time”: 345},
    “has_taken_respect_seminar?”: true,
    // ...
}

Now it’s easy to look up a user’s stats. Just select the_json_string from metrics where user_id = 394. Now you have a JSON string with all the metrics. Now, whenever you add a new metric, you just have your code calculate it and stick it in the JSON object before writing it back as a string. No schema changes, no data model cabal to get through, just code sticking things in JSON. If you decide integer-valued stats should also support a count for the last 42 days, just calculate it and stick it in everyone’s JSON.

But here’s a problem: suppose you want to find all the users with more than 26 hugs given in the last 30 days. How do you query this data? SQL can’t read JSON, so you can’t do select * from metrics where the_json_string.hugs_given.30_days > 26. That JSON type is opaque to the relational database. (Unless you’re using PostgreSQL. We’ll pretend you’re not. The real project I worked on used MySQL, but I believe Oracle and MS SQL Server also lack a JSON type.) So how do you write this query? You can write your own terrible ad hoc JSON parser using regexes and LIKE right there in your database console, or you can write a script in some language like Python or Ruby that reads in the data, parses the JSON string, and does the check for you. Writing this script can also get surprisingly tricky. If the entire table doesn’t fit in memory, you can’t just read all the rows and loop over them. The simple approach would be to get the maximum id of the table and then select each row by id, skipping over misses, until you’re done. But this incurs a disk read on each iteration of the loop, plus possibly a network call if you’re not allowed to run the script right on the database server, so it’ll be agonizingly slow. So you’ll probably read in batches. Now you have to mess with the logic around batch sizes and offsets. It probably won’t take more than an hour or so to write this script. Still, a simple select is something you can dash off about as fast as you can type it in. And even if you write the script generically enough that you now have a general purpose script for querying based on values in the JSON string, you still don’t have all the tools available in one place. If you’re working in the database console and suddenly realize you need to query some data in the JSON string to proceed, you have to stop, go to another window, and run the script. There’s extra friction if you need to feed that data into an SQL query at the database console to move on. There’s this gaping hole where you can’t use SQL to find data that’s stored in your SQL database.

Putting data in strings also kills the validation benefits that a statically typed data model gives you. If someone accidentally writes {"hugs_given": "fish"} to the database, the database doesn’t know that that’s not how you measure hugs. If someone accidentally writes {"hugs_given": {"30_day": 5, "30_day": 10} to the database, the database doesn’t know or care that you repeated a key.

How to Proceed?

So now we know both the strengths and weaknesses of storing JSON in string columns. It’s flexible, easy to change, easy to write, easy to understand, easy to work with in code. On the other hand, it’s meaningless to the SQL database (unless you’re fortunate enough to be working in Postgres). It’s just this string of characters that might as well be the complete text of the classic novel Lady Chatterley’s Lover by DH Lawrence. You can’t query it from SQL in any meaningful way. You can’t put indexes on it. (Well, you could put an index, but it would be massive and not very helpful.) You lose the data validation benefits that a relational schema gives you. What do we do?

It’s possible that you feel you can live with the limitations of JSON strings in database columns. Maybe you can. I have a feeling they’ll eventually get in your way, though. They certainly got in mine.

If you have the option of switching data stores, you could just use Postgres, or some JSON-oriented document store like MongoDB, although I have reservations about that in general. In Crayon Art Online, as in the real application that inspired it, metrics are calculated by a separate process and stored in a different database than the main application, so even if your main application is on MySQL, it might be doable to switch your metrics application to Postgres. (It wasn’t for me, for a variety of boring non-technical reasons.)

But suppose you’re convinced that storing an opaque string of JSON won’t work for you, and you don’t have the option of switching data stores. You’re left with trying to find a better data model to fit this into a relational schema. In this particular case, where the nesting isn’t that deep, it’s not actually that hard to come up with an acceptable solution that’s better than JSON string in a column.

To start with, I’d represent each individual calculated metric as a row with a user id, type tag (hugs_given, positivity_quotient, etc.), a date range tag (30_day, 90_day, etc.), a value, and a created_at date to record when we calculated it.

In my experience, rolling date ranges are a lot simpler than actual dates, so even though it might be tempting to get rid of the string keys for 30_day, 60_day etc. and come up with some way of storing the date range based on actual dates, it could also complicate things a lot. If you need to know the actual date range this covers, you can count backwards from the created_at. If you expect to need this a lot, it might make sense to store the date keys as integers that represent the number of days covered by this metric, so we’d have a column days_covered, and then we can find the start date of this period by subtracting that from the created_at.

The main hurdle with representing this in a schema is the fact that the value type can be an integer, real number, Boolean, string, or possibly other things. Making separate tables for each value type is tacky in my opinion, but you could go this route if you want. Then you could use some kind of wrapper at the data access layer to hide the four classes from your code and just present one consistent interface for them. An even more tacky variant would be to make your value column store a foreign key into another table, and make four value tables that hold your values. So you’d have the five tables metric, int_value, boolean_value, real_value, and string_value, and metric would have a type column that tells it which table to look for its values in, and a value_id column that points into one of the value tables. Not only does this have the same disadvantages as the four metrics tables, it’s also weird and complicated to understand. Another option: store the value, and only the value, as a string, and implement some kind of conversion to the correct type at the data access layer. You can still run queries on the numeric values by casting the string to a number, so you’re not as out of luck as you were with opaque JSON strings. None of these are great solutions. This was the original hurdle that led us to putting JSON in a text column, and I don’t know of any really good way out of it. But I do think there are better ways than JSON in text columns.

This is not a perfect, beautiful schema. It’s not a work of art. It’s not going to get written up in textbooks or research papers as amazing. But it is a real relational database schema which will work better for you in the long run than shoving a string of JSON into a column. (Or a string of CSV, or XML, or YAML, or any other kind of structured text format that isn’t supported natively by the database type system.) You can do queries on it, you can analyze it, you can put indexes on it, you can implement certain data consistency standards on it (e.g. a unique key on type, days_covered, and created_at that prevents you from accidentally inserting the same start for the same date range more than once). There are also opportunities to make things better as you discover problems. With text in a column, there’s not a lot else you can do with it when you discover problems.

This schema is also more flexible and manageable than separate tables for each metric. If you need to add a new stat, just start calculating it and add a new tag for it. If you want to support a new period of time, just make a tag for it and start storing it. No need for schema changes.

The Takeaway

If you came into this as an expert who loves relational databases and strongly typed schemas, I hope you can understand a little more the kind of problems that seem too hard to fit into the strictures of such a schema, driving beginners to want to put JSON in string columns.

If you came into this as someone who’s occasionally—or more than occasionally—thought the answer to all your problems was shoving JSON in a text column, I hope you now understand why that’s not really the answer to your problems. Or you learned that you need to switch to another data store that supports shoving JSON into text columns better, and now every company you work at, you’ll lobby to switch to Postgres or Mongo. Someday that might even lead you to a job as a MongoDB ambassador who gets to travel around to conferences telling everyone why shoving JSON in text columns is terrible, and they should instead shove JSON in MongoDB to solve all their problems. Just think; I could be starting careers here today.

Advertisements

Foxes and Hedgehogs

When I first started programming, which wasn’t actually that long ago, but it feels like ages because of the constant churn, languages were like drunks in bars: they all had a philosophy. Some of them made a lot of sense, like Ruby’s “Principle of Least Surprise”. Some of them were incoherent, like Perl’s “TIMTOWTDI” and “GALFKGAFCVHAJRTOAIADGDLFJKC”. Many were encapsulated in nice little slogans that people could yell at each other in forums. Others were implicit, like Java’s philosophy, “Make Insanely Complicated Libraries So Companies Have To Hire Exorbitantly Priced Consultants To Teach Their Programmers To Accomplish Basic Tasks In Java”. But whether the philosophy made sense or didn’t, whether the philosophy was implicit or explicit, the languages all had one.

The philosophies served a lot of social purposes, but in most cases they did begin as a motto or design principle that the language creators kept in mind when designing the language. Recently, the languages I’ve seen becoming popular, like Kotlin and Swift, don’t seem to have philosophies. They have purposes: Swift’s purpose is freeing iOS and macOS programmers from having to manually decrement reference counts, and Kotlin’s purpose is to integrate with Jetbrains IDEs. But they don’t seem to have language design philosophies the way older languages do.

A lack of a language design philosophy is sort of a language design philosophy. It means the creators don’t have an overriding vision for the language. But without an overriding vision, how do they decide which features get in and which don’t? I don’t know a ton about either Kotlin or Swift, but one thing always stuck out to me in tutorials on them: the tutorials read a lot like lists of features from other languages. “Swift has for-each loops. Swift has lambdas. Swift has classes. Swift has limited type inference.” They would explain how the particular implementation worked, but it was usually substantially similar to other well-known languages. Kotlin and Swift are not random assemblages of features the way languages like C++ and Perl 5 and PHP are. They have an aesthetic, which I’d call roughly “functional-lite”. They’ve settled on a Python-esque profile, largely imperative with object-oriented and functional features worked in around the edges, and added some Java-style static typing. I divined that they choose features based on what they think working programmers will find useful.

Of course, no popular language aimed at working programmers purposely adds features that are highly likely to be useless. But some languages are built according to a strong, overriding vision of what set of abstractions will ultimately prove useful to working programmers. Others are more pragmatic; they add whatever features have been proven, through long use in other languages or through long experience without the feature, to be practical and useful for working programmers. Older languages made this into a philosophy itself: Perl’s “There Is More Than One Way To Do It” was expressing a belief that there should be multiple syntaxes for the same abstraction, presumably because each one would feel natural in different situations. (I never worked with Perl, but the concept does make sense—the same abstraction can manifest concretely in different ways, each of which feel appropriate in different situations.) This distinction is sort of a meta philosophy on top of the various philosophies (explicit and implicit) of different languages. Swift and Kotlin are in the latter camp, but since they’re backed by large corporations and have special niches where their dominance is uncontested, they didn’t need to create marketing slogans that explicitly state it the way Perl did. They didn’t need to differentiate themselves from other languages with marketing; they differentiated themselves by being the language of Apple / Jetbrains and by being the language of iOS development / extreme IDE integration.

Once I realized this divide existed, it reminded me of foxes vs. hedgehogs. The idea comes from a snippet of Ancient Greek poetry attributed to Archilochus: “The fox knows many things, but the hedgehog knows one big thing.” To wit, the fox stays alive with many tricks to keep ahead of its enemies, whereas the hedgehog has one big trick, stabbing its enemies on its spines. (We’re excluding the possibility of a hedgehog that’s the fastest thing alive.) I first heard about it in politics. Some politicians dabble in many issues; they might have a few flagship issues, but they’re conversant with almost all of them, and they’re quick to educate themselves about new issues. Other politicians have one big issue that they care about more than anything, and are less concerned with other issues. Isaiah Berlin applied it to writers in his essay The Fox and the Hedgehog: some writers, like Plato, Dante, and Nietzsche have one big idea that colors all their work, while others, like Shakespeare, Joyce, and Goethe, write about a wide range of ideas and experiences. It’s the same with programming languages. Some languages are fox languages: they might have a few flagship features, but they’ll include various features from all over the place, and while they’re not necessarily garbage heaps of trendy features, they’re more open to borrowing. Some languages are hedgehog languages: they’re built around a singular vision. The vision might be towards a stance on a big philosophical issue. It might be a vision of a language built around a big abstraction. It might be a vision of a language perfectly suited for a single restricted domain. For a hedgehog language, everything that goes into the language serves its one singular vision.

Why is this distinction useful? Because new languages (and libraries, frameworks, platforms, etc., which can be judged much like programming languages) appear constantly, and they’ll all tell you that they can solve all your problems. Evaluating whether the tool’s meta-philosophy matches your goals is a good first step to finding out if that’s true. It’s not about one or the other being bad or wrong. Both have their uses.

If you think hedgehog languages are useless, think about this. Nobody likes writing multithreaded code. Well, okay, some people like writing it, but nobody likes debugging the insane synchronization issues that crop up later. Nobody likes spending a bunch of time crafting the perfect multithreaded architecture only to discover it has a fundamental flaw that makes all your code actually just really expensive sequential code. So a bunch of hedgehogs who had a vision of a better way to do this concurrency thing went off and created abstractions like software transactional memory, actors, communicating sequential processes, single-threaded event-driven programming, and others, and then a bunch more hedgehogs made languages like Clojure, Erlang, Scala, Go, and Node that baked in support for them as part of their visions. Another big victory for hedgehogs: OOP. Hedgehog languages like Smalltalk and Java went all in on OOP and convinced a generation of programmers that writing libraries in C was painful and bad. Whether you think OOP was a good idea or not (I’m starting to have my doubts nowadays), it was a huge propaganda victory for hedgehogs.

I hope it takes less convincing to see that fox languages have their uses. Their benefits are more obvious: programmers who’ve used the same features in other languages can jump right in and understand them without learning new abstractions. Knowledge about how to solve problems in other languages carries over without any trouble. The thing is, these seem to be cyclic. For a while, everyone just uses the fox languages and appreciates all their homey conformity and routine. Want to write a website? Here’s a fox language. Want to write a mobile app? Here’s another fox language that’s basically the same, except maybe you can leave parentheses off no-arg function calls. Want to program an Arduino? Here’s yet another fox language that’s basically the same as the first two, except maybe it has an easier dictionary syntax. Then some new problem appears. Reusable software. Distributed computation. Big data. Machine learning. Whatever. The old fox languages are missing something that makes this new problem easy to solve. So a round of hedgehog languages appears, built around a single grand vision of how to solve this new problem. Sometimes people adopt one of these hedgehog languages at large scale. Sometimes it survives multiple cycles by turning into a fox language. Other times, the new hedgehog languages are too inaccessible or don’t have enough marketing or whatever, but after a while a new round of fox languages appears that brings their features to the masses.

This is where I see Swift and Kotlin fitting in. Everyone was aware that Java 6 was making things harder. Everyone knew Java 6 didn’t have the right abstractions for modern, quick-churn web and mobile code. A round of hedgehog languages came out to address this—-Scala, Clojure, Go. They’ve gotten some adoption. Go is foxy enough, and backed by a huge enough company, that it might live to become a fox language. But the barrier to entry was high enough, and the lack of marketing profound enough, that most of these languages haven’t caught on in a big way. Not like Java, or Python, or PHP have. So now we’re having a round of fox languages—-Swift, Kotlin, Javascript after ES2016—-that consolidate features from all of them. We’re having languages like Java go foxy and add features from these languages to new versions.

So at this point, you might be saying “That’s great, but I’m a programmer who writes boring business apps. How do I figure out what I should be using?” I’m glad you asked! The distinction between fox languages and hedgehog languages is most important of all for you. Boring business apps have a particular set of constraints that are different from most other kinds of program. The code will be changed in a million places in a million small ways over its lifetime. A wide array of programmers should be able to understand the code, and legibility, documentation, and easy modification are the most important characteristics. There is almost never a good reason to use hedgehog languages for boring business applications. Hedgehog languages are based on a grand vision of how computation should be expressed. Boring business applications are not about grand visions of how computation should be expressed. Boring business applications are best expressed in the abstraction of structured programming, which is now such a fundamental part of programming languages that it doesn’t even matter whether a language is for foxes or hedgehogs, it does support structured programming. So, if you write boring business applications and you’re evaluating a language (or framework, database, etc.), you should look for signs of foxiness. Are the features widely supported in other, similar, existing tools? Do the creators stress practicality, usability, reliability? Are the overarching abstractions familiar, or at least widely available in other foxy tools? If you’re seeing a bunch of unfamiliar features, or familiar features mixed in unfamiliar ways; if the creators are talking about vision, about changing the way things are and blowing up the status quo and revolutions and paradigm shifts; or if the tool is based around abstractions that you’ve never seen before or only exist in very obscure tools, then you’ve got a hedgehog tool. It’s much harder to evaluate whether a hedgehog language is suited to what you need. Hedgehogs have to be carefully matched to a use case and a set of problems, because hedgehogs are usually created in response to a specific use case and set of problems.

I’ll end by listing off a few languages and evaluating whether I consider them fox or hedgehog, as they currently stand.

C: At the time it was made, very much a fox language. Now it’s something of a hedgehog language, in that its one big use case is programming in low-level, resource constrained, or high performance contexts. You probably wouldn’t choose to write your startup’s killer new social network in C.

C++: Definitely a fox language. C++ keeps on adopting new features like lambdas that proved popular in other languages. It’s mostly seen nowadays in similar contexts as C, but unlike C, it keeps piling on new features and new abstractions in response to programmer demand.

Java: Java started as something of a hedgehog language. It’s become less of one over time, but still looks spiny enough to me. It was object oriented programming for the masses, C++ without the rough edges, Smalltalk without the flexibility, and its adherence to object orientation still seems a bit militant. It’s gotten new features like generics and lambdas, but it’s always added them by putting a thin layer of syntax on top of an existing part of the language: generics just tell to compiler to automatically insert a cast for runtime, and lambdas are just syntactic sugar for anonymous classes implementing interfaces with one method.

Python: Python is a fox language with a few hedgehoggy principles, like “There should be one and only one obvious way to do it”, which leads to enforced whitespace conventions. Over time, it added features like classes, map and reduce, list comprehensions, iterators, generators, and async / await in response to other languages. It also made it relatively easy to call code written in C, which is the foxiest trick of all.

Python is also a good case study on communities vs. languages themselves. The Python language is a fox, but the community behaves like a bunch of hedgehogs. Python seems to usually resist adding any feature until people have been asking for it for three or four releases, and then someone will make a PEP and it’ll get added.

Ruby and Perl: These two are a lot like Python, but with even fewer overt hedgehog tendencies. They both borrowed things from bash, which is a very foxy language in that you only use when you have no other way to accomplish something.

Scala and Clojure: These two are the opposite of Python, Ruby, and Perl, which were foxy with a few hedghoggy traits. Scala and Clojure are hardcore hedgehogs with a few foxy traits, the biggest one being that they run on the JVM and can interoperate with Java code. Scala has a grand vision of mixing Java-style OOP with Haskell-style functional programming. Clojure has a majestic ambition to be a Lisp dialect for the modern era, mixing a solid concurrency story and good support for hash maps with the functional features of Haskell and the homoiconic power that Lisp is known for. Both of them have high learning curves due to lots of unfamiliar abstractions and unusual features, like Scala’s implicits and Clojure’s menagerie of types for managing state under shared memory concurrency.

Lisp in general is a hedgehog language. It has a grand vision of all programming as lists of lists that directly represent an abstract syntax tree. Common Lisp is a Lisp that tried to be foxy with stuff like rplaca and the progn macro. Scheme is a Lisp that skewed further towards the hedgehog. Clojure is somewhere in the middle; it runs on the JVM and adds features like hash maps, sets, and regex literals that are foxy compared to Scheme, but also encourages pure functional programming and immutability to an uncomfortable hedgehoggy extent. Its concurrency stuff is pure hedgehog—yes, even core.async. We’ll get into this in the next section.

Go: I’ve only used Go a little bit, but to my eyes it’s a language that lures you in by appearing to be a fox, but then when you go to pet it, surprise! You’ve got hedgehog spines stuck in your hands! It’s a hedgehog that knows how to play tricks like a fox.

There are two aspects to Go’s inner hedgehog. One is its concurrency stuff, with goroutines. Yes, I know it’s based on communicating sequential processes, from the 70’s. The thing is that there are at least two whole generations of programmers who’ve grown up never hearing about communicating sequential processes and never using a language that supported it. There’s the 1990s era of big enterprise Java developers, and there’s the current era of hipster startup Node hackers, plus a few intervening generations, all united in their never using a language that supported CSP as a first class abstraction. Since Clojure’s core.async is a ripoff of goroutines (as admitted by Rich Hickey himself), it’s also a hedgehog trait.

The other aspect is Go’s slavish adherence to C’s precedent. It has pointers. It has printf and scanf. It doesn’t have generics, and has repeatedly refused requests to add them, and justified this decision by appeals to Go’s design philosophy of simplicity. Whenever a widely requested feature is refused because of a design philosophy, that’s a sure sign of a hedgehog. I’m not saying whether that’s right or wrong, but it is a fact. C itself was foxy, but Go’s insistence on copying it beyond reason is total hedgehog.

My Biases

To disclose my own biases, I actually like hedgehogs, except when they go wrong, which they can, terribly, much more easily than foxes. Go’s hedgehog nature was what induced me to try it over something more foxy like Node or Kotlin. Clojure’s hedgehog nature induced me to try it over something more foxy like Scala. (Yes, Scala is also a hedgehog, but it looked deceptively like a fox from far away.)

Hedgehog languages are usually the ones that win over the people, but don’t get any traction in industry, and forums are filled with programmers wondering why no one can see how great this language is and what a killer secret weapon it would be for some team.

Even though I like hedgehogs, when the chips are down and the title is on the line, I always go back to foxes. If I have to program something and I have no idea what I’m doing, it’s back to Python. I don’t reach for Clojure or Go, because with hedgehog languages, you never know when you’ll run into some problem that, due to its very nature, doesn’t translate well into that language’s idioms and abstractions. If it turns out that a bunch of hideous index manipulation is the only way to solve the problem, I’d rather take my chances with Python than write weird Clojure code full of atoms and swap! and reset!.

Java is a hedgehog language. And a lot of pain in modern software has happened because people used Java somewhere where its insistence on object-oriented programming, combined with the militant adherence to object-oriented design principles that its programmers have developed, didn’t do a good job representing the problem domain. I work on a Java codebase at my job, and every day, I run into things that Java makes harder by being such a freaking OOP hedgehog. Testing, for example. You can write testable code in Java, but features like private access and lack of standalone functions are hostile to it, not to mention all the ways Java developers’ culture has evolved to make it harder to write testable code. This has led to tons of cultural conventions and weird libraries to make writing testable code possible, but still not easy. If the chips were down and the title was on the line, I would never turn to Java. It’s fairly likely if I pick Clojure that I’ll have to do some sequence manipulation that’s hard to translate into point-free applications of map, reduce, and filter, but it’s just as likely if I pick Java that I’ll end up wanting arbitrarily nested maps, or pairs, and have to run off and write new classes to represent them. And then I might want a 3-tuple. Guess what? Another new class!

On the other hand, when you have to explore a gigantic monolithic codebase written by 50 programmers over five years, Java’s not so bad. It forced the other programmers to explicitly write out everything, explicitly represent everything with classes and statically typed variables, and even tag which methods you need to worry about seeing again outside of that class. It’s all about knowing when to pick the right tool. Evaluating foxiness vs. hedgehoggosity is one way to get a feel for a tool. Then you can decide whether that particular hedgehog’s spines will solve your problem, or if you need the cunning of a fox to get through.

You Won’t Be Remembered For Your Code

A lot of people who are first getting into the software industry seem to think that they’re going to make some kind of impact with their code. The dumber ones think it’s going to be some kind of huge splash, like people are going to print their code on giant wall scrolls and put it up in museums so future programmers can come through and admire its structure and that amazing little trick with the lambda. The smarter ones know it will be a small impact, but they still think it will be an impact of some kind.

Allow me to rain on your parade: you will never be remembered for your code. Code in industry software is endlessly erased and rewritten. Even that picture of a penis you carved into the wall of a bathroom stall in sixth grade and then initialed like it was some great work of art is more likely to still be there, because replacing the walls of bathroom stalls is expensive and laborious. But erasing your clever little trick with the lambda and replacing it with ten lines of plodding that even the biggest dunderhead couldn’t possibly fail to understand takes about five minutes, and people who erase code are on salary, so it didn’t even cost the company five minutes of work, whereas people who replace bathroom stall walls get paid hourly. Plus, now even the biggest dunderhead couldn’t possibly fail to understand that code, so they can fire the programmer who replaced your clever little lambda trick and hire the biggest dunderhead for a lower salary.

No one is remembered for code. Steve Jobs is remembered for product design and marketing. Bill Gates will be remembered for being insanely rich. Programming languages are products, so programming language creators are also remembered as product designers, not for their wonderful code, and they’re easily forgotten once no one is using the language anymore; give it twenty years and Larry Wall will be a random piece of programming trivia alongside Jon Ousterhout, Jean Ichbiah, and Niklaus Wirth. Same goes for frameworks, which are even more disposable than programming languages. If anyone remembers David Heinemeier Hansson or Rails at all, it won’t be for some ingenious metaprogramming trick he used in Active Record.

And these are the really famous people, the people whose code actually affected the way thousands of others do their jobs. Even they will not be remembered for how wonderful their code was. So trust me when I say that you won’t be remembered for your great code.

However, if your code sucks, it will keep your memory alive for a while. Every time someone goes into a sloppily formatted file with dozens of unnecessary global variables, lots of overlong methods, and comments that say x++; // Add one to x, they’ll head on over to Git and find out who wrote this crap. If you wrote enough of the code, you might even become something of a legend for a while. Of course, the other programmers will make a point of extinguishing your imprint from the code, so your code is still headed for death, and your legend will live on only in the tales the other programmers tell of the crappy code they suffered with until they could sneak in enough refactoring to eliminate it. You’ll be remembered, but you’ll be remembered the same way Tom Green is remembered, as an irritating bore that everyone raced to expurgate as quickly as humanly possible.

So don’t get into software if you think code is a good way to be remembered. It’s not. Product design is more likely to get you remembered by someone, but that’s not foolproof either; not many people, even in programming circles, have heard of Ward Cunningham, who invented the wiki, or Dan Bricklin, who invented the spreadsheet, even though those are both products at least as important as anything Steve Jobs ever made.

JSON in Java: A Case Study in Pain

Dealing with JSON in most dynamically typed languages is dead simple: you have a function that can inflate a string into the language’s native hash map type, and you have another function that can serialize the language’s native hash map type into a string of JSON. These languages all support nested maps, some sort of list, and strings, numbers, and booleans well enough to automatically map from JSON’s simple data types into the language’s native types.

Dealing with JSON in the emphatically statically typed, the holdover that’s still hanging around like a bad hangover, the one, the only Java will eventually make you wish you were dead.

I’ll be up front with you: this isn’t going to be a deep or insightful read. It’s mostly going to be a rant, with lots of hate for Java and static typing, but mostly static typing as seen in Java, because I’ve never tried to work with JSON in languages like Haskell with more sophisticated type systems, so you Haskell lovers are free to stay smug in the knowledge that you work with a better class of type system, nay, not a class in sight, but rather a category. If you love Java and static typing, you might want to tune out, because I am not at all interested in hearing for the hundredth time how dynamic languages force you to write unit tests for the stuff that static languages check at compile time. This is also not meant as a top-to-bottom indictment of static typing; it’s an examination of how static typing utterly fails to solve one specific problem in an elegant way.

To start with, I’m going to present the problem. Then I’ll examine what I find to be the root causes of this problem, and then rant a little about some features Java will never get that I think could help this problem.

The Problem

There are two major things you need to be able to do with JSON in your programming language: turn a string of it into your language’s native data types so you can work with the data, and turn your language’s native data types into a string of it so you can send it across a network.

JSON—Javascript Object Notation—is based on a restriction of Javascript’s native data types. Looking at the standard, we see that it supports objects, arrays, strings, numbers, booleans, and null. On the face of it, it seems like we should be able to represent these types with native Java types: maps for objects, lists for arrays, and Java’s string, number, boolean, and null for those types. But maps and lists in Java have to be restricted to a single type, so to represent JSON values like {"status": true, "message": "Status normal"} or [23, 'hello', false], we have to use a Map<String, Object> and a List<Object>.

Unfortunately, this almost immediately leads to problems. Java lists and maps don’t support being stringified as valid JSON, so you have to write custom code to do it. Actually writing this code isn’t too hard as long as you remember that JSON supports arbitrarily nested lists and objects, so you have to make sure your serialization is applied recursively until you bottom out. But it has all the usual disadvantages that come when you write custom code for fundamental operations instead of getting it from a library: you have to test it, support it, and maintain it yourself, and it’s harder to bring into new projects since you can’t just let your build tool grab it from Maven Central.

Working with Objects in Java is rather painful; if you want to do anything with them, you have to cast them. This requires you, the programmer, to know what type every one of those objects actually is. Look at this rather ugly bit of code:

Map<String, Object> jsonMap = getJsonMap();
List<Object> jsonList = jsonMap.get("theList");
if ((Boolean)jsonList.get(2)) {
    Integer idNumber = (Integer)jsonList.get(0);
    User user = getUserById(idNumber);
    user.setPreferredGreeting((String)jsonList.get(1));
    user.save();
}

Not only is this code ugly, it also loses the much-vaunted type safety of Java. You could get runtime type errors if it turns out that the first item of jsonList is not actually an Integer or the second item is not actually a String.

We can solve the ugliness problem a bit by throwing out the built-in Java map and list and using a special set of classes to represent JSON data types. This is what org.json or Jackson and GSON’s tree structure use. Instead of a Map, we’ll use a JsonObject; instead of a List, we’ll use a JsonArray; and instead of using Java strings, numbers, and booleans, we’ll use a JsonPrimitive. We’ll make all of these classes implement the JsonType interface.

The JsonObject class will behave like a Map<String, JsonType>. The JsonArray will behave like a List<JsonType>. The JsonPrimitive will store its value as a string, or possibly an Object. It will have a set of three methods: getAsNumber, getAsString, getAsBoolean, and possibly corresponding isNumber, isString, isBoolean. We glossed over the extra complication that the Javascript/JSON number type can be either integral or floating point; we also glossed over the extra complication that JSON numbers don’t have a specified size, so what we get could potentially overflow an Integer or Double. We could handle those problems here either by making getAsNumber return a BigDecimal or making separate getAsInteger, getAsDouble, getAsBigInteger, getAsBigDecimal methods. Or we could decide that overflow will be rare enough that we’ll tolerate errors when it happens, and make getAsNumber return a Double.

With this new set of classes, we can rewrite the above example:

JsonObject root = getJsonMap();
JsonArray array = root.get("theList");
if (array.get(2).getAsBoolean()) {
    User user = getUserById(array.get(0).getAsInteger());
    user.setPreferredGreeting(array.get(1).getAsString());
    user.save();
}

It’s at least less ugly without all those casts and temporary variables, and since the types make it obvious that we’re dealing with JSON, we don’t have to name everything jsonWhatever. But we’ve still lost type safety; if we do array.get(2).getAsBoolean() and that item isn’t a boolean, we’ll still get a runtime type error. And these custom JSON types are inevitably less convenient to work with than the standard library data types. What options do we have if we want to iterate over a JsonArray? Does it implement Iterable<T>, which will let us use the “smart for-loop”, or are we stuck with the C-style for-loop? Can we get the entry set of a JsonObject and iterate over all its keys and values if we want to? Do these classes implement Collection<T>, making them streamable?

This solution with separate JsonWhatever classes is the quick and dirty way of solving the JSON problem in Java. Imagine that your car breaks down in the middle of the desert. You have a tarp, a roll of duct tape, and a long pole in the back, so you tape the tarp to the pole, tape the pole to your car, and you’ve got a sail. The JsonWhatever classes are the homemade car sail of JSON handling. It does nothing whatsoever to fix the fundamental problem, but it does kind of get you moving again, in a way that’s slightly better than walking or pushing the car.

Most high-powered JSON libraries in Java go with a more sophisticated approach, borrowed from ORMs such as Hibernate. The Java language really really wants you to specify everything precisely, categorically, and at a low level: this class has these exact five fields with this five exact names, and this one’s an Integer, and this one’s a String, and this one’s a NosehairInspectorFactory, and so on. So why not create a class that specifies what your JSON will look like, precisely, categorically, and at a low level?

This is what Jackson and GSON do. They include a class, called ObjectMapper in Jackson, that can take a normal Java object and turn it into a string of JSON, or take a string of JSON and a normal Java class and create an object of that class from the string. Here’s an example:

class MyObject {
    private Integer myId;
    private String myPreferredGreeting;
    private Boolean isHasBeenGreeted; // Note, this is what you get when you insist on starting all boolean names with "is".
    private MyOtherObject myOtherObject;
    private List<Integer> allMyBuddiesIds;

    // All that getter and setter nonsense that Java insists on...
}

Jackson can map between this JSON and an object of that class:

{
    "myId": 23,
    "myPreferredGreeting": "Whassup dorks?",
    "isHasBeenGreeted": false,
    "myOtherObject": {
        "myFavoriteDrink": "Rolling Rock",
        "myFavoriteFood": "Cheese pizza with the crust cut off",
        "myLifePhilosophy": "YOLO bros"
    },
    allMyBuddiesIds: [21, 33, 49]
}

You might be wondering exactly how it does that. The answer is magic. As previously mentioned, I hate magic. But Jackson is magic from top to bottom. Except when it’s not.

The basic idea, to give a little more detail, is that Jackson can parse a JSON string into some intermediate form (which may or may not be the tree model it also offers, its version of our JsonWhatever types above; I haven’t looked into Jackson’s source code far enough to know). From there it can use reflection, possibly assisted by annotations you put on your class, to look for a way to map from the JSON object to your class. If you’re familiar with dependency injection, it’s pretty much the same idea: Jackson looks for a constructor or some setter methods it can use to inject the values from the JSON into an object of the class you provide. Then we have a regular Java object, referred to in this context as a POJO (“Plain old Java object”) since it’s not a special JSON-whatever object.

This sounds amazing. And it is pretty amazing, for JSON with all primitive types and no nesting. Unfortunately, Jackson doesn’t play well with generic types, so you end up having to use raw types for maps and lists, bringing us back to our original problem where we have to know what types are stored in maps and lists and cast them when we extract them.

The other problem with this is that some APIs return JSON responses that look like this:

{
  "statuses": [
    {
      "coordinates": null,
      "favorited": false,
      "truncated": false,
      "created_at": "Mon Sep 24 03:35:21 +0000 2012",
      "id_str": "250075927172759552",
      "entities": {
        "urls": [

        ],
        "hashtags": [
          {
            "text": "freebandnames",
            "indices": [
              20,
              34
            ]
          }
        ],
        "user_mentions": [

        ]
      },
      "in_reply_to_user_id_str": null,
      "contributors": null,
      "text": "Aggressive Ponytail #freebandnames",
      "metadata": {
        "iso_language_code": "en",
        "result_type": "recent"
      },
      "retweet_count": 0,
      "in_reply_to_status_id_str": null,
      "id": 250075927172759552,
      "geo": null,
      "retweeted": false,
      "in_reply_to_user_id": null,
      "place": null,
      "user": {
        "profile_sidebar_fill_color": "DDEEF6",
        "profile_sidebar_border_color": "C0DEED",
        "profile_background_tile": false,
        "name": "Sean Cummings",
        "profile_image_url": "http://a0.twimg.com/profile_images/2359746665/1v6zfgqo8g0d3mk7ii5s_normal.jpeg",
        "created_at": "Mon Apr 26 06:01:55 +0000 2010",
        "location": "LA, CA",
        "follow_request_sent": null,
        "profile_link_color": "0084B4",
        "is_translator": false,
        "id_str": "137238150",
        "entities": {
          "url": {
            "urls": [
              {
                "expanded_url": null,
                "url": "",
                "indices": [
                  0,
                  0
                ]
              }
            ]
          },
          "description": {
            "urls": [

            ]
          }
        },
        "default_profile": true,
        "contributors_enabled": false,
        "favourites_count": 0,
        "url": null,
        "profile_image_url_https": "https://si0.twimg.com/profile_images/2359746665/1v6zfgqo8g0d3mk7ii5s_normal.jpeg",
        "utc_offset": -28800,
        "id": 137238150,
        "profile_use_background_image": true,
        "listed_count": 2,
        "profile_text_color": "333333",
        "lang": "en",
        "followers_count": 70,
        "protected": false,
        "notifications": null,
        "profile_background_image_url_https": "https://si0.twimg.com/images/themes/theme1/bg.png",
        "profile_background_color": "C0DEED",
        "verified": false,
        "geo_enabled": true,
        "time_zone": "Pacific Time (US & Canada)",
        "description": "Born 330 Live 310",
        "default_profile_image": false,
        "profile_background_image_url": "http://a0.twimg.com/images/themes/theme1/bg.png",
        "statuses_count": 579,
        "friends_count": 110,
        "following": null,
        "show_all_inline_media": false,
        "screen_name": "sean_cummings"
      },
      "in_reply_to_screen_name": null,
      "source": "Twitter for Mac",
      "in_reply_to_status_id": null
    },
    {
      "coordinates": null,
      "favorited": false,
      "truncated": false,
      "created_at": "Fri Sep 21 23:40:54 +0000 2012",
      "id_str": "249292149810667520",
      "entities": {
        "urls": [

        ],
        "hashtags": [
          {
            "text": "FreeBandNames",
            "indices": [
              20,
              34
            ]
          }
        ],
        "user_mentions": [

        ]
      },
      "in_reply_to_user_id_str": null,
      "contributors": null,
      "text": "Thee Namaste Nerdz. #FreeBandNames",
      "metadata": {
        "iso_language_code": "pl",
        "result_type": "recent"
      },
      "retweet_count": 0,
      "in_reply_to_status_id_str": null,
      "id": 249292149810667520,
      "geo": null,
      "retweeted": false,
      "in_reply_to_user_id": null,
      "place": null,
      "user": {
        "profile_sidebar_fill_color": "DDFFCC",
        "profile_sidebar_border_color": "BDDCAD",
        "profile_background_tile": true,
        "name": "Chaz Martenstein",
        "profile_image_url": "http://a0.twimg.com/profile_images/447958234/Lichtenstein_normal.jpg",
        "created_at": "Tue Apr 07 19:05:07 +0000 2009",
        "location": "Durham, NC",
        "follow_request_sent": null,
        "profile_link_color": "0084B4",
        "is_translator": false,
        "id_str": "29516238",
        "entities": {
          "url": {
            "urls": [
              {
                "expanded_url": null,
                "url": "http://bullcityrecords.com/wnng/",
                "indices": [
                  0,
                  32
                ]
              }
            ]
          },
          "description": {
            "urls": [

            ]
          }
        },
        "default_profile": false,
        "contributors_enabled": false,
        "favourites_count": 8,
        "url": "http://bullcityrecords.com/wnng/",
        "profile_image_url_https": "https://si0.twimg.com/profile_images/447958234/Lichtenstein_normal.jpg",
        "utc_offset": -18000,
        "id": 29516238,
        "profile_use_background_image": true,
        "listed_count": 118,
        "profile_text_color": "333333",
        "lang": "en",
        "followers_count": 2052,
        "protected": false,
        "notifications": null,
        "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/9423277/background_tile.bmp",
        "profile_background_color": "9AE4E8",
        "verified": false,
        "geo_enabled": false,
        "time_zone": "Eastern Time (US & Canada)",
        "description": "You will come to Durham, North Carolina. I will sell you some records then, here in Durham, North Carolina. Fun will happen.",
        "default_profile_image": false,
        "profile_background_image_url": "http://a0.twimg.com/profile_background_images/9423277/background_tile.bmp",
        "statuses_count": 7579,
        "friends_count": 348,
        "following": null,
        "show_all_inline_media": true,
        "screen_name": "bullcityrecords"
      },
      "in_reply_to_screen_name": null,
      "source": "web",
      "in_reply_to_status_id": null
    },
    {
      "coordinates": null,
      "favorited": false,
      "truncated": false,
      "created_at": "Fri Sep 21 23:30:20 +0000 2012",
      "id_str": "249289491129438208",
      "entities": {
        "urls": [

        ],
        "hashtags": [
          {
            "text": "freebandnames",
            "indices": [
              29,
              43
            ]
          }
        ],
        "user_mentions": [

        ]
      },
      "in_reply_to_user_id_str": null,
      "contributors": null,
      "text": "Mexican Heaven, Mexican Hell #freebandnames",
      "metadata": {
        "iso_language_code": "en",
        "result_type": "recent"
      },
      "retweet_count": 0,
      "in_reply_to_status_id_str": null,
      "id": 249289491129438208,
      "geo": null,
      "retweeted": false,
      "in_reply_to_user_id": null,
      "place": null,
      "user": {
        "profile_sidebar_fill_color": "99CC33",
        "profile_sidebar_border_color": "829D5E",
        "profile_background_tile": false,
        "name": "Thomas John Wakeman",
        "profile_image_url": "http://a0.twimg.com/profile_images/2219333930/Froggystyle_normal.png",
        "created_at": "Tue Sep 01 21:21:35 +0000 2009",
        "location": "Kingston New York",
        "follow_request_sent": null,
        "profile_link_color": "D02B55",
        "is_translator": false,
        "id_str": "70789458",
        "entities": {
          "url": {
            "urls": [
              {
                "expanded_url": null,
                "url": "",
                "indices": [
                  0,
                  0
                ]
              }
            ]
          },
          "description": {
            "urls": [

            ]
          }
        },
        "default_profile": false,
        "contributors_enabled": false,
        "favourites_count": 19,
        "url": null,
        "profile_image_url_https": "https://si0.twimg.com/profile_images/2219333930/Froggystyle_normal.png",
        "utc_offset": -18000,
        "id": 70789458,
        "profile_use_background_image": true,
        "listed_count": 1,
        "profile_text_color": "3E4415",
        "lang": "en",
        "followers_count": 63,
        "protected": false,
        "notifications": null,
        "profile_background_image_url_https": "https://si0.twimg.com/images/themes/theme5/bg.gif",
        "profile_background_color": "352726",
        "verified": false,
        "geo_enabled": false,
        "time_zone": "Eastern Time (US & Canada)",
        "description": "Science Fiction Writer, sort of. Likes Superheroes, Mole People, Alt. Timelines.",
        "default_profile_image": false,
        "profile_background_image_url": "http://a0.twimg.com/images/themes/theme5/bg.gif",
        "statuses_count": 1048,
        "friends_count": 63,
        "following": null,
        "show_all_inline_media": false,
        "screen_name": "MonkiesFist"
      },
      "in_reply_to_screen_name": null,
      "source": "web",
      "in_reply_to_status_id": null
    },
    {
      "coordinates": null,
      "favorited": false,
      "truncated": false,
      "created_at": "Fri Sep 21 22:51:18 +0000 2012",
      "id_str": "249279667666817024",
      "entities": {
        "urls": [

        ],
        "hashtags": [
          {
            "text": "freebandnames",
            "indices": [
              20,
              34
            ]
          }
        ],
        "user_mentions": [

        ]
      },
      "in_reply_to_user_id_str": null,
      "contributors": null,
      "text": "The Foolish Mortals #freebandnames",
      "metadata": {
        "iso_language_code": "en",
        "result_type": "recent"
      },
      "retweet_count": 0,
      "in_reply_to_status_id_str": null,
      "id": 249279667666817024,
      "geo": null,
      "retweeted": false,
      "in_reply_to_user_id": null,
      "place": null,
      "user": {
        "profile_sidebar_fill_color": "BFAC83",
        "profile_sidebar_border_color": "615A44",
        "profile_background_tile": true,
        "name": "Marty Elmer",
        "profile_image_url": "http://a0.twimg.com/profile_images/1629790393/shrinker_2000_trans_normal.png",
        "created_at": "Mon May 04 00:05:00 +0000 2009",
        "location": "Wisconsin, USA",
        "follow_request_sent": null,
        "profile_link_color": "3B2A26",
        "is_translator": false,
        "id_str": "37539828",
        "entities": {
          "url": {
            "urls": [
              {
                "expanded_url": null,
                "url": "http://www.omnitarian.me",
                "indices": [
                  0,
                  24
                ]
              }
            ]
          },
          "description": {
            "urls": [

            ]
          }
        },
        "default_profile": false,
        "contributors_enabled": false,
        "favourites_count": 647,
        "url": "http://www.omnitarian.me",
        "profile_image_url_https": "https://si0.twimg.com/profile_images/1629790393/shrinker_2000_trans_normal.png",
        "utc_offset": -21600,
        "id": 37539828,
        "profile_use_background_image": true,
        "listed_count": 52,
        "profile_text_color": "000000",
        "lang": "en",
        "followers_count": 608,
        "protected": false,
        "notifications": null,
        "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/106455659/rect6056-9.png",
        "profile_background_color": "EEE3C4",
        "verified": false,
        "geo_enabled": false,
        "time_zone": "Central Time (US & Canada)",
        "description": "Cartoonist, Illustrator, and T-Shirt connoisseur",
        "default_profile_image": false,
        "profile_background_image_url": "http://a0.twimg.com/profile_background_images/106455659/rect6056-9.png",
        "statuses_count": 3575,
        "friends_count": 249,
        "following": null,
        "show_all_inline_media": true,
        "screen_name": "Omnitarian"
      },
      "in_reply_to_screen_name": null,
      "source": "Twitter for iPhone",
      "in_reply_to_status_id": null
    }
  ],
  "search_metadata": {
    "max_id": 250126199840518145,
    "since_id": 24012619984051000,
    "refresh_url": "?since_id=250126199840518145&q=%23freebandnames&result_type=mixed&include_entities=1",
    "next_results": "?max_id=249279667666817023&q=%23freebandnames&count=4&include_entities=1&result_type=mixed",
    "count": 4,
    "completed_in": 0.035,
    "since_id_str": "24012619984051000",
    "query": "%23freebandnames",
    "max_id_str": "250126199840518145"
  }
}

That’s from Twitter’s API reference. Now imagine writing a Java class to represent that response. Now imagine doing it again for whatever other insane responses Twitter wants to fling at you.

Jackson has what feels like hundreds of bizarre annotations and configuration options that let you control the translation process between a JSON string and your class, or get around problems like JSON responses suddenly getting new fields and mutually referential objects causing stack overflow errors due to infinite recursion. It’s a huge, complex library, and while there’s plenty of information out there, it can be hard to word a search query in the right way to find a solution to your specific problem. At my company, we briefly looked at migrating from org.json to Jackson, but it was going to be a massive amount of work to figure out the right magical incantations to make Jackson not break and then either go mark up all our database objects with the right annotations or write new view-model classes to use for JSON translation. And since none of us were Jackson experts, we had a steep learning curve ahead of us.

Imagine your car breaks down in the middle of the desert, but you happen to be right next to an abandoned auto shop with a set of books and tools. You don’t know the first thing about cars, other than where the gas goes, but you figure you’ve got a good brain, so you sit down with a book and start reading. After six months of alternately reading and tinkering on some of the forgotten wrecks left on the hydraulic lifts of this abandoned auto shop, you’re ready to start on your car. You eventually get it kind of working, though it can’t go above 37 mph, makes a weird stuttering noise as you drive, and you have to stop every 15 miles and go tighten some bolts, and also you just spent six months camped out in an abandoned auto shop in the middle of the desert. Here you’ve addressed the fundamental problem that your car is broken, and what you came up with is definitely better than a homemade car sail, but was it really worth that six months you spent in that abandoned auto shop?

The root cause, and how to fix it

The root cause of all these problems is Java’s static typing. Well, to be fair, it’s also Javascript’s dynamic typing. JSON is based on Javascript, so it’s perfectly fine with lists of mixed types, maps of string to whatever you please, and arbitrarily nested self-referential objects. Languages like Python, Ruby, PHP, and Clojure, which are dynamically typed and have a similar set of primitives to Javascript, are also fine with that, so you can trivially map any JSON object to a native data type in those languages.

Java is not fine with any of that. It demands lists of a single type and maps of strings to a single type, and it asks that you please tell it beforehand just how deep you’re going to nest those self-referential objects. You can cheat your away around this by declaring the type as Object, but working with Objects is tedious, ugly, and loses you the only advantage to having all these restrictions.

How can we fix this? Here a few features that would definitely help. They’re listed in order of how likely they are to be added to the Java language, starting at “Maybe in Java 26” and ending at “You’ve got a better chance of convincing your company to rewrite everything in Erlang”.

Optional dynamic typing

C# supports a type keyword dynamic. This makes a variable dynamically typed; the compiler basically pretends that it can be any type and supports any operation, and allows a runtime exception to be thrown if you do the wrong thing with it.

This would, at the very least, make the JsonWhatever strategy a lot easier: instead of needing a JsonPrimitive with all those conversion methods, just make the JsonObject and JsonArray store values of type dynamic.

This would also make Jackson’s problem with generic types easier to deal with: instead of binding to lists of Object or maps of string to Object, it can bind to a list of dynamic or map of string to dynamic.

This approach still gives up type safety, but it makes giving up type safety a lot easier to work with and less painful.

Union types

Languages like Haskell and ML support something called union types. Essentially you can define a new type which can hold a value of any of a given set of types.

Imagine that Java had a syntax like type JsonVal = Map<String, JsonVal> | List<JsonVal> | Integer | Double | String | Boolean | null to define a new type called JsonVal. This declares that a JsonVal can be any of those listed types, but no other. If our code ever tries to assign a FizzingWhizbeeFactory to a JsonVal, the compiler will complain.

This can buy us both freedom from casting and arbitrary nesting. The compiler has a list of possible types, so it doesn’t need us to tell it exactly which one a given value is; if we call a method on a JsonVal, it knows where it can look for that method, and if it can’t find the method anywhere, it can flag a compile-time error. It doesn’t need us to cast for it. And the JsonVal type is defined recursively: a JsonVal can be a Map<String, JsonVal> or a List<JsonVal>. This gets us the arbitrary nesting we need to accurately represent JSON.

There are other challenges to getting union types right in Java. For instance, if I have a union type type MyType = A | B and C is a subtype of B, can MyType be an instance of C? Can I cast an instance of C to B and assign it to MyType? Can I then downcast an instance of MyType which is assigned an instance of B to C if that’s what it really is? Or do we just do what generics did, throw the whole mess out, and say that any type which isn’t explicitly listed in the declaration of the union type is invalid? But if we could get union types in Java, it would go a long way towards solving our JSON problem in a type-safe way.

A Dedicated JSON type

This is the easiest option for programmers, and the most deranged from a language design standpoint.

Scala, Visual Basic, and Kawa (a dialect of Scheme on the JVM) have XML literals. In Scala, for example, you can write this:

val people = 
    <people>
        <person firstName="Ted" 
                lastName="Bundy" 
                gender="M" />
        <person firstName="Jeffery" 
                lastName="Dahmer" 
                gender="M" />
        <person firstName="Charles" 
                lastName="Manson" 
                gender="M" />
        <person firstName="Rasmus" 
                lastName="Lerdorf" 
                gender="M" />
    </people>

So in Java, we could just modify the language to let you write this:

Json someJson = {
    "people": [
        { "firstName": "Adolf", "lastName": "Hitler" },
        { "firstName": "Idi", "lastName": "Amin" },
        { "firstName": "Pol", "lastName": "Pot"},
        { "firstName": "Hiro", "lastName": "Mashima"}
    ]
}

The Json type can have a static method, Json.fromString, that inflates a string into JSON, and its toString method can be overridden to serialize a Json into a string.

Scala can sort of do XML literals as part of the normal language, because Scala’s design is already deranged. In Java, the language would have to be modified. But there is precedent for this: arrays in Java are already just random tacked on garbage with special syntax and special behavior that applies to no other part of the language. And Java’s principal use today is as a server-side web programming language; given how important JSON is to web programming, it might be worth modifying the core language to add this.

Still, this is the least likely of the options. Adding lambdas and method references was a massive deal, and those had already proven their worth in dozens of other languages over a period of almost fifty years. Java is not a language that likes revolutionary features; it prefers to wait until every language in the universe, even C++, has a feature, and then panic and squeeze something on top of the existing language features that’s just good enough to disunite the opposition.

Conclusion

Dealing with JSON in Java is quite painful, and it’s because JSON is inherently dynamically typed due to its heritage in Javascript’s native object type, and Java’s static type system is too rigid to accommodate it in an elegant way.

Since JSON is used for communicating between systems and components of a system, the difficulty of working with JSON in Java can affect how you choose to design your API responses. You might choose to favor flat objects over nested, as we used to do at my company. Instead of something like this:

{
    "room": {
        "name": "library",
        "carpetType": "Persian rug"
    },
    "killer": {
        "name": "Colonel Mustard",
        "weapon": "lead pipe"
    }
}

You might choose this:

{
    "roomName": "library",
    "roomCarpetType": "Persian rug",
    "killerName": "Colonel Mustard",
    "killerWeapon": "lead pipe"
}

In Java, with Jackson, the second form requires a single class with four strings. The first form requires a class with two subclasses, each with two strings, and probably a few annotations to make sure everything happens correctly when you bind from a string. So up front, the first form is more complicated to work with in Java, and this extra complexity just gets more pronounced as your JSON responses get more complex.

But using the second form causes other problems. Since JSON keys have no guaranteed order, you could send the response to a client like a browser and have it print with the keys all randomized, which makes the response much harder to read if you need to debug. If a client of your API only cares about the room, they have to loop over all the keys and find the ones that start with room instead of being able to just grab the room object. You can write that annoying code every time, or you can write some stupid little helper function that you have to document and test to loop over a JSON object and find all the keys with a certain prefix. You’ll probably also have to duplicate this function across various code bases in your stack. This tradeoff might be worth it if you just have a bunch of Java code communicating (e.g. an Android app and a Spring API), but if you have other languages in your stack, like Javascript or Python, you’re forcing them to conform to the lowest common denominator.

If you’re writing web code nowadays, JSON is a fact of life, and it’s a great transmission format: it’s simple, readable, and powerful, easy to parse, able to represent almost any kind of data in human readable format. It’s elegant and easy to use, a big improvement in a lot of ways over RPC formats like COM and CORBA, unreadable binary formats, and XML. It’s a travesty that Java has such terrible support for something so elegant, widespread, and fundamental.

Web Programming: A Prophecy

Over the course of the 20th and 21st Centuries, a lot of jobs that used to require human labor have been turned over to robots. Factory jobs are going to robots. Mining jobs are going to robots. Fighter jet and spy plane pilot jobs are going to robots. Retail jobs are going to robots, with self checkouts and restaurants like Eatsa.

I used to believe, smugly, that programming was too complex and difficult to ever be taken over by robots. When I read an article by some business commentator that claimed programming jobs were going to be this generation’s equivalent of the high-wage factory jobs of the 1950s and 1960s, I scoffed. Until I actually did some of what passes for programming in industry.

It’s pretty much all been lamented before: modern programming in a corporate environment is mostly writing little bits of glue code to combine a bunch of libraries written by some distant genius. Mike Taylor’s version was the first version of this lament that I saw, back when I was still in school and under the impression that industry programming was at least a little bit like personal or school programming. But I’m not here to lament. I’m here to make a prophecy.

In my current job, I am a full-stack web developer. I write the server-side code. I write the client-side code. I write the HTML and CSS. I create the database schema and write the SQL scripts to update the database. Is it complex and difficult? Kind of. But it’s complex and difficult because I have to have passing familiarity with Java, Javascript, HTML, CSS, and SQL, and work through all the amazing ways they don’t fit together well. My part in the whole mess is to drop in fairly simple, monotonous bits of boilerplate code to grab things from the database and insert them into an HTML document. Then I manipulate another mess of frameworks and libraries to show it all nice and pretty and make dynamic behavior on the page. Between the server-side framework, the Javascript framework, Bootstrap, and various other libraries, about 90% of the work is done by magical frameworks that just use little spludges of repetitive code I write to get that last 10% of the way towards becoming a web app.

I find it extremely hard to believe that there’s no way to automate that last 10%. There has to be a tool that makes it possible for someone to drop in those repetitive little globs of boilerplate just the way they like, and get a web application out of it.

As a full-stack web developer, I also offer some advice on design, UX, and business needs. But I’m not a designer, a UX pundit, or an expert on the business needs. My degree is in computer science. My expertise is programming. But there’s damn little programming in the code I write, and nothing I’d call computer science. The main hurdle is the delicate little interconnections between these rickety, rigid frameworks, the brittle wooden bridges I have to cross by writing repetitive boilerplate. But, per hypothesis, there has to be a way to create a tool that automates that repetitive boilerplate. Without that, there’s no programming left in creating the sort of custom line of business app that I work on. You could get together a designer, a UX pundit, and an expert on the needs of the business, and have them use this tool to create whatever web app they want. There are actually already tools that are close to this. Squarespace. Drupal and Joomla. WordPress is a little closer to full-stack web development, but I see job ads for “Wordpress programmers” rather than “PHP programmers” or “Front end programmers”, which suggests to me that WordPress is a specialized tool. As I was writing this I discovered WebML, which already sounds pretty close to what I was envisioning.

Once I realized that, I saw how right Mr. or Ms. Business Commentator whose name I’ve forgotten was in his or her comparison of software developer jobs to factory jobs. Right now, tools like Squarespace and Joomla still have certain limits, and there’s cachet in having a custom web app, so everyone and their dog needs a full-stack web developer to make one for them. Trucking companies need web apps; banks need web apps; government agencies need web apps; churches need web apps; even the Amish probably have a full-stack web developer on their payroll to make an Amish web app for them. (Job perks include hot apple Brown Betty in the kitchen, bonuses paid in handmade furniture, and a company carriage for the commute.) But once this magical tool whose future existence I’ve postulated comes to pass, none of these people will need full-stack web developers. Except the Amish; they need someone who’s okay being in a building with electricity every day. But everyone else will just have business experts make the web app. This might be the product manager, or the head of the department that’s going to use the web app, or the lady down the hall who knows Excel and VBA really well and consequently always gets stuck doing everyone’s spreadsheets. But there’s no reason to bring a programmer into the company and pay them a bunch of money and have them learn the business requirements when you can just have someone who already knows the business requirements use the tool to build the web app.

This doesn’t mean that all programming jobs will suddenly disappear. There will still be programmers. You can’t have a business person build something with a prepackaged tool when you don’t have any business requirements and no one’s ever built anything like it before. Google-scale stuff isn’t going anywhere. Google-level programmers aren’t going anywhere. But people like this guy are going to lose their jobs, or they’ll be slotted in at a lower pay grade to build web apps with the new tools. Even people who aren’t like that guy, people who don’t obstinately hold it as a personal tenet that no one should know more advanced technical subjects like Big O and data structures and we should all just concentrate on understanding business requirements and spelling Dijkstra’s name right, will lose their jobs in droves. They’ll lose their jobs because they’ll have spent careers working on simple little full-stack web apps built on existing frameworks instead of on Google-scale stuff, and suddenly businesses won’t need that anymore, or at least they’ll be paying a lot less for it.

So how long do we, the generic full-stack web developers working on dinky little web apps, have before the apocalypse? My guess is no more than ten years. The tools are already getting there, and they keep advancing. With single-page apps and faster connections, users are already being trained to expect more lag in web apps, so the kind of performance tuning that an experienced developer does will become less relevant. And businesses would love to be able to fire a bunch of expensive programmers and squeeze a little more work out of the business people. They’d love to be able to stop stocking soda and Terra chips for us arrogant, overpriced keyboard jockeys.

A few questions

At this point, you might be thinking my prophecy is wrong and the era of anyone who knows Javascript and one server-side framework being able to find jobs anywhere at relatively high salaries will last forever. And you might be right. The odds are against it, but you might be right.

However, I’m going to try and preemptively address a few questions I can see people having about this prophecy.

Won’t programmers just switch to programming the tools?

No. Well, yes, some of them will. But there aren’t nearly enough jobs working on the tools for everyone who now works as a generic full-stack web dev to get one. The world only needs two or three competing tools in each space, but even if a dozen more manage to scrape by, the scale won’t match the number of people currently working on dinky custom web apps alone for non-tech companies. Think about it: have you ever written a framework? And if you have, did anyone use it? Even though it sometimes feels like there are new frameworks for Javascript coming out on a daily basis, only about three or four actually get widely used, and most framework creators don’t get paid to work on them. The ones that do get used, the ones whose creators do get paid to work on them, are the ones like Angular and React that are backed by large tech companies.

Writing the tools will be the kind of Google-scale programming that will hang on after the Twilight of the Generic Full-Stack Web Devs. But for every former generic full-stack web dev who switches to working on tools, there will be at least ten who get shut out because of significantly lower demand for programmers, and probably another five who switch to using the new tools to do their own jobs at a significant pay cut just to stay employed. Companies in all industries needing to have their own custom web app and hiring a generic full-stack web dev to glue together some frameworks for them has created a supply of generic full-stack web devs that won’t be nearly as much in demand once business people can make the dinky custom business app.

Isn’t programming too complex and difficult to automate like you describe?

I hope if you’ve done the kind of generic full-stack web development I’m talking about here, you know that’s not true. The frameworks already do most of the work. Disorganization and the dismal state of web standards is the only thing holding us back from a single isomorphic client/server framework that lets us build monolithic web apps in a single language the way we used to build desktop apps. And with Node.js, client-side MVC frameworks, and Bootstrap, we keep getting closer to that one framework to find them all and in the DOM bind them.

There are other kinds of programming that I don’t foresee being automated so easily: hardware programming, big data, machine learning, cryptography, and of course the tools such as compilers, IDEs, frameworks, servers, and databases. Programming those things is complex and difficult, and programmers working on those things will continue to have lucrative careers. This prophecy will largely affect people not working in the tech industry, but working on web apps (and possibly mobile apps) in other industries.

Didn’t this already fail?

There have been various attempts to create tools that business people could use to make custom applications, e.g. CASE tools. Many failed. But one, COBOL, turned out to be one of the most successful programming languages of all time.

It’s possible that efforts to create the super tool I’m prophesying here will also fail. But I think modern day has one big thing that makes such a tool more likely to thrive: Google. Older tools were expensive proprietary things, and you had to read a gigantic manual to figure out how to use them. But nowadays, if you need help, you can Google for answers, or you can ask a question on a Stack Exchange site. And there will probably be blogs and tutorials and help guides for the tools, and maybe even public online documentation by the manufacturers, the way Oracle does with Java.

If you’re a busy business expert trying to make a web app, this makes a huge difference: instead of having to spend hours reading through a giant manual and days figuring out the best way to use what your tool gives you, you can just Google for an answer and copy it verbatim, like programmers do.

I’m a generic full-stack web developer and I believe in your prophecy. What should I do?

As a prophet of doom, I sadly don’t have any constructive suggestions for you, other than to consider where your career might go after this doom comes to pass.

If you enjoy the technical side of things, you can try to become an expert in one of those other areas of programming that won’t be wiped out in this apocalypse. If you find cryptography or machine learning too imposing, go for stuff like dev ops and database adminstration. Or if you’re more of a front end person and you like design or UX, you could head in that direction.

Alternately, if you like business requirements and management, like this guy, go full speed in that direction. I don’t, so I won’t be joining you.

What are you planning to do once the prophecy comes to pass?

Once my career in generic full-stack web development dries up, I’m going to take a few months to backpack around Europe and find myself. In a small Italian village on the Amalfi coast, I will rescue a stunningly beautiful woman from three drunken Sigma Chi boys on Spring Break from Penn State. The only daughter of the owner of a struggling trattoria, she will have the heart of poet and the sensitive wit of a Regina Spektor lyric. Our love will blossom like the last flowers of June, and after months of sexual tension and exquisite trattoria food, we will finally consummate our relationship. But the next morning, I’ll receive word that my beloved Albanian bulldog, Bullard, has died. Stricken with grief, I will weep tears of manly remorse, and rue that I was not at Bullard’s side in his final hours. I will return to the United States to further weep, and to visit his grave under the shade of a poplar tree in my parents’ backyard. But in the ashes of this tragedy a fire will kindle within me, and I’ll start a popular YouTube channel in which I talk about the wonders of Italy and Albanian bulldogs while wearing an oilskin duster in burnt sienna, a color that brings out my eyes. As my channel gains subscribers, I’ll start a companion blog and podcast and soon become a widely recognized blogger and YouTube celebrity.

So you see, I’ve got this all figured out. The end of generic full-stack web development will ultimately be a good thing for me. Maybe it can be a good thing for you too.

Every Freaking Tech Job Ad Ever

We’re looking for a superstar COBOL ninja to join our team! Applicants must have at least 40 years experience with COBOL. COBOL.NET experience preferred. You’ll spend every day hacking away on our exciting, disruptive legacy banking product, making it more robust and scalable as well as more exciting and disruptive. We are an exciting, young, hip, radical, groovy team of hackers, rockstars, ninjas, and wizards who love tech, startups, and tech startups, as well as disruption, excitement, and disruptive excitement, particularly in tech! We’re changing the face of banking by using the latest, most exciting technologies to disrupt all the old ideas about storing money and replace them with exciting, disruptive new ones! So come join us and remake the face of banking by making hacky temporary workarounds to problems with our mainframe OS that were fixed thirty years ago by upgrading to Windows 3.1!

Requirements:

  • 40 years COBOL experience.
  • Experience working with decimal computers
  • Teletype experience
  • Experience with mainframes
  • Experience with banking
  • Experience with banking mainframes
  • Experience with exciting, disruptive developments in legacy banking mainframes
  • PhD. in Computer Science, Computer Engineering, or Nucular Engineering
  • 10+ years software experience
  • Must be under 24 years of age
  • Must be able to lift up to five grams
  • Great team player who loves working in teams and can’t do anything alone. Anything.
  • Be a hip, radical, groovy ninja who’s excited about disrupting banking with disruptive legacy mainframe systems

Nice to have:

  • Experience with TCL, MUMPS, or RPG
  • Experience with Visual Foxpro
  • COBOL.NET
  • Dataflex experience
  • Juggling three or more balls
  • Knowledge of wilderness survival
  • Willing to spend business trips living in a van down by the river
  • A pulse

Introducing MonkeyType.js!

When I needed to choose a Javascript framework for a work project, I put a lot of thought into it. I tried Angular, Angular 2, Ember, React, Redux, Backbone, Dojo, Vue, Mithril, Marionette, CanJS, Knockout, Polymer, and js_of_ocaml. None of them worked. They all just felt wrong.

What I eventually realized is that these frameworks felt wrong because they don’t use the right abstraction. Modeling web apps as MVC, MVVM, MVP, MVT, MVASDFASDHSFGSDFGHQRTQRTAFGLKJNNGJSKL, these are all the wrong abstractions.

That’s why I decided to create my own Javascript framework. Instead of using broken models of abstraction like Model-View-Controller or Model-View-View Model, it uses MOT: Monkeys On Typewriters. And I called its name MonkeyType.js.

I know what you’re thinking: what does a stupid front-end engineer who took on $157,000 in student loans to follow his passion and get a degree in Lithuanian literature with a minor in Cloud Shape Interpretation Studies, laid around on his mom’s couch for five years, got an online Master’s degree in Motivational Speaking, worked as a tutor for left-handed children, decided to enroll in divinity school, dropped out after accruing another $34,000 in debt, lucked into a plum job at a grocery store making $15 an hour where his only real duties were printing signs for the manager to stick in the break room, learned HTML, CSS, and Javascript after the manager asked him to make a website for her side business selling purebred blue fawn Azawakhs (also known as the Tuareg Sloughi according to the AKC), found out you could make money with this Javascript stuff, and finally moved out of his mom’s house, know about designing a framework? Way ahead of you! MonkeyType.js is based firmly on MATH! It uses the celebrated Infinite Monkey Theorem, described by famed French measure theorist Félix Édouard Justin Émile Borel in 1913, to created a model of UI firmly based on MATH!, with grounds in MATH!, concretely resting on a foundation of MATH! to create a uniquely appropriate abstraction coming, like all the greatest and most appropriate abstractions, from MATH!

The Infinite Monkey Theorem states that given infinite monkeys banging the keys on infinite typewriters for an infinite length of time, the probablity that Hamlet will appear somewhere on one of the monkey’s typewriters is equal to 1, i.e. one of the monkeys almost surely produces Hamlet. MonkeyType.js takes this theorem to heart and uses it to create an abstraction for UI programming that’s so wrong, it couldn’t be more right. The best part about a client-side framework based on the Infinite Monkey Theorem is that the Infinite Monkey Theorem is MATH!, and uses terms like almost surely in confusing ways that only make sense if you know MATH!, and are otherwise just confusing and misleading! You have to be really smart to know MATH!, so you also have to be really smart to understand MonkeyType.js. No dummies who only know how to make solid, functional user interfaces that are intuitive and aesthetically pleasing; only the best front-end engineers need apply to use MonkeyType.js, because with MonkeyType.js, you have to be able to do all of those things, and also know MATH!, and not dumb kiddy stuff like fractions, but REALLY HARD MATH! that famous French guys invented. Everyone knows French stuff is really cool and hard to understand; the other night I watched Amelie, and I had no idea what was going on, but it was French, so I can just go up to people and say “Hey, have you seen Amelie? It’s French” and they know how smart and cool I am to be able to understand French stuff. And then they’re even more impressed when they find out I’m the inventor of MonkeyType.js and I understand MATH!

A typical MonkeyType.js program has two parts: the monkeys, and the typewriters. The monkeys represent the data. They’re sort of like the model in other frameworks, except with more MATH!, and more French. The reason monkeys are a better abstraction for data than a model is that models are stupid and for babies. When I was a stupid baby I got this model of an airplane for my birthday. It was wood. I kept trying to build it and getting splinters in my fingers that really freaking hurt. Finally I got a splinter under my fingernail, and it hurt so bad that I got mad and threw that dumb airplane model across the room and it broke, and my mom had to come in and hug me. I wouldn’t stop crying, so finally my mom had to take me to the zoo to make me quiet down. I didn’t stop crying until I saw the monkeys. I was only fifteen years old, but I kept that love of monkeys my whole life, and I never cried again, until I got home and stepped on a broken piece of the airplane model and got a splinter in my foot.

The typewriters represent the appearance, sort of like the view in other frameworks, except with more MATH! Typewriters are way better than views. I hope I don’t have to explain this one; views remind me too much of 1985’s A View to a Kill, and then I spend all day mumbling “Dance into the fire” under my breath and my smarmy co-workers who pretend to be my friends but secretly laugh at me because they all hacked into the Pentagon’s computers when they were ten years old and I didn’t learn to program until I was forty-seven start talking behind my back.

The monkeys pound on the typewriters to create the website. Unlike other dumb frameworks that have something like a controller standing between the model and view, MonkeyType.js lets the monkeys pound directly on the typewriters. Here’s where the really cool part comes in. Obviously we can’t have infinite monkeys and infinite typewriters on a modern computer with finite memory, so MonkeyType.js uses a clever optimization called a monkey barrel. Instead of infinite monkeys, the monkey barrel contains about twelve or thirteen monkeys that it keeps recycling. Every time you ask for a monkey, the monkey barrel gives you a monkey. It might be the same monkey you got last time. It might be a different one. Since one monkey’s as good as another monkey, with this approach, you effectively have infinite monkeys. The typewriters are stored internally in a circular buffer, which gives the impression that there are infinite typewriters, but it’s actually just, like, twelve or thirteen typewriters on a loop. And to create the impression of infinite time, MonkeyType.js is coded in an excruciatingly dumb way so that all the pages hang for at least a minute before doing anything, and when they do do something, they just load a spinner that swirls for another minute or two before the actual page you were looking at finally loads. This impression of infinite time is increased even further in production enviroments, where hundreds of trackers and advertising services also get to load before your content. This makes MonkeyType.js the perfect framework for rich media sites like Buzzfeed, Huffpost, and ZergNet, where every page is so obviously loading five hundred ads before it even thinks about delivering the content you asked for, because the advertising is actually the content and the content was actually just a bait-and-switch to get you to look at ads. But who cares, because we deliver ads with a firm basis in MATH!

In conclusion, where other JS frameworks give you broken abstractions based around dumb ideas like being easy to learn, easy to write, or easy to use to create great UIs, MonkeyType.js gives you abstractions based on MATH! and awesome stuff like monkeys and typewriters. Who cares if it causes memory leaks in your browser? It’s MATH! I hope you’ll consider MonkeyType.js for your next project, and finally learn what a real abstraction looks like.