This was originally posted on my Tumblr while I was still on hiatus.
I am absolutely terrified as all shit about sharing this right now, but I think it’s good for me if I do. So I am. I’ve never really made anything I’ve done (as a programmer) in to something public-facing before. It’s easy to talk yourself out of showing other people what you’re working on. It’s a defense mechanism, because criticism can be terrifying. But I promised myself that I would release something that I’ve done outside of work to the populace before the end of the semester, so showing this off is the first step towards that end.
This is a project that I’ve been working on, on-and-off, for over a year. It’s come together rather clumsily, with some input and hands-on work from some coworkers and friends (including fellow Tumblr’r Andrew), but this has really been an interesting side project that I’ve been toiling away on tirelessly with what little spare time I have in addition to two semesters of AI credits.
This is Wernicke (or at least that’s what it’s being called right now), named after Wernicke’s center in the brain, responsible for speech comprehension. It’s a collection of moving parts all geared together with the end goal (which I’m still pretty far away from) of allowing for rapid prototyping and simulation of automated smart-homes with a coactive (.pdf link) Natural Language interface.
Under the hood, it’s several different components:
A dynamic REST API built on top of Sinatra, whose endpoints grow and shrink dynamically based on the current configuration of the “home”. The Sinatra server maintains the state of the system as well as managing incoming and outgoing commands. Most importantly, the Sinatra server is Arduino-aware, and can actually have effect on the real world (though this is currently not being used, I tested it almost a year ago and I plan on wiring up some new hardware before the end of the month). I call this the Cortex. It receives RESTful instructions, and orchestrates the system’s “end effectors”.
A Natural Language text processing engine, written in Haskell and the initial focus of the project. Unfortunately, it’s probably the one unit that has received the least love; this is not a testament to missing functionality, but rather a testament to how easy it was to get this part working and how hard it was to get all of the other stuff wired together sanely. Between now and April 26th, this is the portion of the project that will be seeing the most development. This is the Engine. It receives input as plain text, and translates the instructions in to commands that target the API endpoints exposed by the Cortex.
A simulated home floor-planner, very spartan and feature-bare, but whose state is dictated entirely by the Sinatra server. It acts as a visualization proxy in the absence of Hardware, and also provides a facility for informing the server of the topology of the home. This is the Sim, and it receives instructions from the Cortex as well as sending it information about home that it is to manage.
A plugin for SiriProxy, that is currently very naïve but acts as a bridge in between Siri and the Engine, so that spoken word can be used as a method for generating input to the Engine.
On April 26th, I’ll be presenting this at my University’s Scholar’s Symposium as my final grade in my AI directed study, and so I have in between now and then to clean up the new GUI and really ratchet up the SiriProxy plugin and the NLP Engine.
I will also be open sourcing all four modules the project on GitHub at that time, as it’s currently under lock and key in a private BitBucket repo.
Wish me luck.