If you've been keeping tabs on me lately, you've probably picked up that I have a new project I'm working on:
DownJS. I'm not going to get into the details of DownJS (that'll be in another post), but there are some things that are blog-worthy about the DownJS system. One of my goals with DownJS was to write the entire site - backend and portions of the frontend - in Node.js. I've been pretty crazy about Node.js for awhile, but have never had the gusto to tackle a full-featured product entirely in javascript. There's been a lot of
hiccups learning along the way, and I just published the biggest of the modules I've used in DownJS. Multipost is a Node.js module for sending data via the multipart/form-data protocol. That probably doesn't make much sense, so read more after the break.
Raise your hand if you've ever submitted something via a web form. Hopefully everyone has their hand raised. If not, scroll down and leave a comment, then come back and raise your hand. OK - put your hands down. The obvious statement here is that pretty much everyone uses web forms, and everyone uses them because they're
so easy. Put in a couple lines of HTML, handle the $_POST variables in PHP, and let the browser do the rest. It's a beautiful thing - at least until you need to send data from a script.
The usual scenario is that an API will offer a standard cURL interface, so you can just use simple cURL syntax for uploading from a script. In DownJS's cases, though, the API we're using for storing the javascript files
requires a multipart/form-data formatted request. This requires all kinds of crazy syntax about Content-Types and Content-Length, and then some weird gibberish stuff called "boundaries". And, for heaven's sake, don't put a \r\n in the wrong place! Even after you figure out the crazy formatting, there's still one big issue that needs to be solved - multipart/form-data (I'm going to call them MFD from now on) requests require an accurate Content-Length header. This means that you normally would have to save the file to the local disk, read it into memory and tack it on to the end of your POST string, check the length, and then send out the entire thing in the HTTP request. Unfortunately, I don't have that kind of memory that I can play with, so a better solution was needed.
Enter Multipost. Multipost is a Node.js modules that allows you to cut out all the gibberish involved in MFD requests, and additionally allows you to pipe files directly over HTTP, rather than loading them into memory beforehand. The idea is that you write some standard javascript syntax, and we handle the mess and send off the request in the most efficient manner. You can check out the source as well as some documentation on the
Github page, and you can easily install Multipost via
NPM.
Multipost is brand new, and relatively untested. I don't necessarily encourage you to use it in a production environment (although, we are at DownJS), until you've thoroughly tested it in your environment. If you find any issues, please drop them on the
issue tracker, and I'll fix them as quickly as I can.
Big thanks to onteria for this wonderful post that guided me through formatting MFD requests.