Arc Forum | A faster Arc startup time (4x better)

Arc Forum

A faster Arc startup time (4x better)

12 points by palsecam 5573 days ago | 4 comments

Ladies and gentlemen,

MzScheme is awesome. Thanks to it, I've succeeded to make the Arc startup time 4 times faster, making it start under 1 second on my machine, which was what I considered an impossible challenge.

OK, let's go.

Easy step, requiring no modification/adding to any file:

   $ mzc ac.scm brackets.scm

This generates bytecode version of these 2 modules, in ./compiled. And MzScheme being awesome, it checks for compiled modules in this directory and use them automatically if present. And (not totally sure), it'll recompile them on-the-fly when they've changed!

This leads to:

   $ time10.pl 'rlwrap mzscheme -f as.scm < /dev/null'  # BEFORE
                     avg    max     min
   real            4.418   4.470   4.360
   [...]
   10 executions.

   $ time10.pl 'rlwrap mzscheme -f as.scm < /dev/null'   # AFTER compil
                     avg    max     min
   real            3.770   3.830   3.720
   [...]

Now, stop here if you satisfied enough, and don't want to change anything to your Arc installation. Else, let's go a little dirty.

First easy step was to compile Arc files to .scm ('acompile in ac.scm). Then (finding a way to) bytecode-compile them. This is more difficult, 'cause `mzc' works better when compiling modules, but nothing is impossible. But this leads to some dirt, 'cause I've found nothing better than creating a new, crappy, as-fast.scm file, and use it instead of the vanilla as.scm. And using a Perl script. And calling it in my arc(.sh) launcher script.

OK, so in as-fast.scm, I directly indicate the .zo (bytecode compiled) Arc files to load (no more use of libs.arc).

The Perl script mimics the very clever behaviour of MzScheme and will re-compile on-the-fly the .arc files when they've changed.

The .sh launcher calls the .pl script (the overhead is something like 0.012 sec when nothing has changed, so not a problem), before to call `mzscheme'.

I'm going to post the .pl and as-fast.scm in comment, but here are the stats:

   $ time10.pl 'arc < /dev/null'
                     avg    max     min
   real            0.938   0.980   0.920
   [...]

Oh gosh I'll be able to code in Arc now. And script in Arc now!

Thanks to jcw, tc-rusho, erikb, conanite and all the others I shamefully forget for helping me understand I should not be a total moron to complain about Arc (startup) terrible perf, since I am not alone. Guys, may you enjoy this patch, because it is for you!

2 points by palsecam 5573 days ago | link

as-fast.scm:

   (require mzscheme) ; promise we won't redefine mzscheme bindings

   (require "ac.scm") 
   (require "brackets.scm")
   (use-bracket-readtable)

   (parameterize ((current-directory (current-load-relative-directory)))
     (load "compiled/arc_arc_scm.zo")  
    ; yeah, should chdir to ./compiled/ directly, but don't want 
    ; to fight w/ MzScheme now. And yeah, the filenames are dirty
     (load "compiled/strings_arc_scm.zo")
    ; (load "compiled/pprint_arc_scm.zo")
    ; Commented, 'cause bug in mzc or smthg (at least on my machine):
    ; compiled/pprint_arc_scm.zo::2989: read (compiled): ill-formed code [...]
    ;
    ; (aload "pprint.arc")  
    ; ^ and this commented 'cause it'd make `arc' take more than 1s, + unused by me.
    ; Uncomment if you need it, my machine is an eeePC, so on a Core 2 Duo
    ; or the like, I suppose you'll still be under 1 sec. No cheat here
     (load "compiled/code_arc_scm.zo")
     (load "compiled/html_arc_scm.zo")
     (load "compiled/srv_arc_scm.zo")
     (load "compiled/app_arc_scm.zo")
   ; Ah yeah, and I don't load prompt.arc, since a long time, useless to me.
   ; Add it here and in alc.pl, loading it will not kill perf I think (loading
   ; bytecode is fucking fast on this fucking good MzScheme).
   )

   (let ((args (vector->list (current-command-line-arguments))))
     (if (not (empty? args))  ; cmdline args are script filenames
         (for-each (lambda (f) (if (string=? f "-") (tl) (aload f))) args)
         (tl)))

Ah yeah, and 'tl2 in ac.scm needs to get back the ':a' stuff (thanks rntz for pointing this is out on the "Arc usable on Unix" thread):

   [...]
   (if (eof-object? expr)
     (when interactive? (display "(quit)") (newline))
     (if (eqv? expr ':a)
       'done
       (let ((val (arc-eval expr)))
    [...]

alc.pl is 56 lines of code, not copied here, grab it at http://dabuttonfactory.com/res/alc.perl

arc(.sh):

   #!/bin/sh

   ## arc: launcher for Arc. Put in your $PATH.

   ARC_DIR="/home/paul/arc/3.1/"  # change this

   alc.pl $ARC_DIR;
   rlwrap mzscheme -qr ${ARC_DIR}as-fast.scm $@

Note you have no manual work to do when modifying one of arc.arc or the libs, alc.pl is good enough to detect changes and do all the work for you (.arc -> .scm -> .zo). But, and this sucks, if you want to add the loading of, say prompt.arc, you should take care to modify alc.pl and as-fast.scm. And also no work if you hack ac.scm, 'cause as previously said, mzcheme will detect this.

Note all of this is not dependant of my previous patch to make Arc OK on Unix, but you'll have to change some things by yourself if you don't use this patch.

Note 1 sec is still very long. Compare to the 0.012 secs of alc.pl.

There is this quote "On the Internet, if it's not instant, it's too long." Well, it's true in computing in general. I still have to wait before the Arc prompt to show, and this sucks.

Note you should not delete libs.arc or as.scm because alc.pl use them to bootstrap.

-----

1 point by rntz 5573 days ago | link

Interesting. I remember arc2-anarki doing something very similar to this, but I don't think it went nearly this far - I think it just precompiled arc.arc to scheme code. I may take a look at seeing whether I can incorporate some of these speed improvements into arc3-anarki, but I'd rather not rely on hardcoding what files get loaded at startup, especially since anarki modifies libs.arc to autoload things in load/, which is quite useful; nor would I like to introduce reliance on Perl.

Personally I just keep an arc session running at all times from within emacs, which I keep running forever, so this doesn't impact me much, but I can see how it would be necessary to use arc for any sort of scripting. Unfortunately I'm returning to college soon, so I don't know whether I'll be able to get around to working on this.

-----

1 point by palsecam 5573 days ago | link

> nor would I like to introduce reliance on Perl.

Your choice, but I noticed the Anarki arc.sh startup script is bash-dependant, which is actually less widespread than Perl. Not totally sure, but the xBSD doesn't have it by default, where they have Perl. Perl is everywhere (even on Windows, see ActivePerl), seriously. Similarly, nearly Linux distro have it by default, where, same thing, some doesn't have bash (I'm sure of this, because I got bad surprises because of this in the past, although don't remember which ones. Can check if you want). Bash is a big fat cow.

Personal opinion, but a shell-script bash-dependant is just killing the shell-script idea. I code shell scripts when I need to be really sure any Unix will make it run, and this implies making it strictly Posix-sh compatible.

> I'd rather not rely on hardcoding what files get loaded at startup

Yes this is a big sucking point. But there is a workaround:...

> especially since anarki modifies libs.arc to autoload things in load/

... then makes it modify as(-fast).scm instead :-P. Or even libs.arc, that just would contains autoloaded stuff (and not the "core" libs).

> be necessary to use arc for any sort of scripting

Yep, but not only. Not only at all. This is related to my way of working, but I'll expose it nevertheless so that people can understand my need for fast app startups, because I know a lot of people like me. I, of course, can understand the "eternal session" stuff, I sincerely find it smart, but really, I know very few people that can stick with it (I can't). I mean, for what I know, you're really not the majority.

I'm actually OK with a script taking a long time (I sometimes like scripts to take time, it gives me the impression it's doing a lot of work and that I'm smart, I'm making the computer do the stuff :-D + allow me to pause for a second => think before typing). It's not OK when I'm in an trial/error process, but even there it's not so bad if I loose some time. Always pleasant to see the computer work, even for nothing.

But I need to spawn/terminate, spawn/terminate processes, and this is have to be quick. Because I've always at least 2 or 3 virtual desktops (eeePC, so it's not "real" vdesktops, I need nearly one vdesktop by app on this tiny screen. Change vdesktop by window, same thing), and needing to find the one in which I started the process just plainly sucks my balls (seriously; even if it takes 2 secs: useless stuff, waste of time). Say for a terminal, I just hit ctrl-shift-enter, a new xterm is spawn in 0.1 sec and I can do what I have to do right now. Same for my web browser, which is Conkeror (or Chrome on Windows but same here, because it starts very quickly), and can be launched in daemon mode so that when you call it, it's there in < 1 sec (and this is why I can't consider the fat Firefox). Same for everything, or should be. For instance, I actually plan to learn vim because Emacs is too long to start. I plan to drop bash, because with the smart completion feature, it's not instant to start and for a shell this is just a sin.

I'm a big procrastinator, and when I want to do some work, this is not often, and if the computer makes it difficult (slow to start), well I come back to procrastinate. Seriously. Even 1 sec can suffise. And this bugs me a lot, because computers nowadays, WTH they've got GHz of power, this should not happen.

I insistate on all this not because I think this is the right way, but because you (I mean, not you rntz personally, I mean people in general) are crazy to ignore this.

Yes, 0.1 sec makes a difference. I'm sure you know about the Google experiment to try to load x * 2 instead of x (don't remember the exact numbers) results by page, the load time increase was something as insignifiant as +0.1 sec, and they actually loose a real % of customers. (If you don't know about this XP, tell me, will go seek for the link, it's very very interesting). People - just - respond - to - speed. I already said that, but Google made his fucking huge empire based merely on this single point. And they continue to expand their empire applying this recipe (Chrome, Google OS which, I bet everything on this, will be 10x faster to start than Ubuntu, GMail on mobile that just indicates "Download GMail for your mobile - it's blazing fast", etc.)

So, but if you think Google is crap, you're crazy to ignore the speed issues.

FTR, if an app can't be fast, good solutions include applications that would not allow multiple-instances, but will come back in focus when called again. Another good thing is, for terminal, tilda/yakuake. Another interesting idea is a client/server idea, where the client is lightweight where the server, launched as a daemon, is long to start (the evsrv started with this idea).

Last point, eternal session is good, but if this means you keep your personal computer on for days, you're just killing the planet, and this obviously sucks. There is this old maxim "Don't waste, even cheap ressources like computer power/electricity, but to not waste human brain power". Gosh I was reading lately how Amazon was applying this, to the point where they removed the light bulbs from their snack distributors, because they're basically useless. Same thing, Amazon is a giant now, so don't think this is stupid (funny story, Amazon also communicated lately on how 200ms (!) of extra latency makes them loose lots of money).

-----

1 point by rntz 5573 days ago | link

> Your choice, but I noticed the Anarki arc.sh startup script is bash-dependant, which is actually less widespread than Perl.

Good catch. I've changed arc.sh so that it should be POSIX sh compliant, but I'm unsure precisely what this requires and don't have the patience to devour the entire spec at the moment. I've tested it against bash and dash thus far. You can give it a look over at http://github.com/nex3/arc/blob/master/arc.sh if you want.

> Last point, eternal session is good, but if this means you keep your personal computer on for days, you're just killing the planet, and this obviously sucks.

Nope, I use tuxonice/hibernate. :)

-----