Did you mean in some other language? It's easy to build a test harness, it's usually the first thing I do with a new language, and invariably just a few lines of code.
(It's not a normal C++ project. I use readable diff directives inside :(..) to add the test harness to the skeleton program at https://github.com/akkartik/wart/blob/cc70d66ee6/literate/00.... But now that this is done, any function I write with 'test_' is automatically run in test mode. Look at the makefile to see how I do that with minimal code.)
Anyways, tell me what language and I'm sure we can get you quickly past this hurdle.
Honestly, there are some fiddly bits about the unit test framework I don't like, but mainly I wanted to write one.
I actually applied with it for Lisp In Summer Projects (http://lispinsummerprojects.org/), which is why I haven't announced it -- you're supposed to do the work yourself, without help. And people here like to help out and post code. :-p
Luckily, at this point it's got the main features I want, so I can actually use it.
Sweet. I had been wanting to play around with the unit test code too -- i'm excited to see what you've put together.
The oauth utility is also for the LISP contest -- we'll see how far I get over the next few weeks.
Either way I'm planning on uploading a few bits and pieces to Anarki or my own repos over the next couple of weeks. (spent some time on anarki's web.arc, the state machine stuff, oauth, some lazy evaluation stuff, etc...)
Please let me know what you think -- email in profile, comment here, open bitbucket tickets, find me on the street^1, etc.
[1] Actually, after writing this, I read your profile, and found you're in Hacker School. I'm in nyc too -- we should meet up sometime. Shoot me an email.
That makes sense :) I'd love to hear more about what's fiddly about the existing version (I have different versions at http://github.com/akkartik/arc, etc.) and why you need the features (suites, nested suites, failure messages, anything else?)
I'm going to have to go to bed soon, as I need to wake up in eight hours and twenty minutes, and I've promised myself I'm going to try to sleep enough, for once.
So I'll just explain what, in my mind, is the biggest difference -- how I want to use it. To run the anarki test-iso test, you execute the entire `(test-iso ..)` sexp. If you want to run a bunch of tests, you have to execute all the sexps.
That's kind of a hassle. Especially if you find a bug, have a bunch of tests that fail, then change a small thing in the function, and want to re-run all the tests.
In my unit-test.arc, all you have to do is call `(run-suites suite-name)`, and it'll run as many tests as you've got in `suite-name`. You don't have to copy a bunch of sexps into the repl or reload the file (and what if you want to run a subset of a file? You can't). Also -- and this is one of the features I'm currently working on (https://bitbucket.org/zck/unit-test.arc/issue/21/after-runni...), what if you run one hundred tests at once? Do you really want to parse -- with your eyes, like a bloody scribe -- every single line of output to find the seven tests that broke? And when you then make a fix, you're not going to want to parse them again, so you'll only run the seven that failed before. So if you broke something else, you won't find that out.
So, what falls out of my desire to run a set of tests easily and repeatably, and have summarized output? Some sort of grouping, with a single point of entry to run the tests -- that is to say, test suites.
Please correct me if I've missed any feature of Anarki's test framework. I did read its code and try it, but I didn't look into other files for supporting functionality.
Thanks! You're absolutely right, I've had every single one of these problems, I just never focused on them. But I'd often comment out a bunch of tests in the middle of debugging just one.
