Arc Forumnew | comments | leaders | submitlogin
arc2c : new version, very soon on the git
11 points by sacado 5838 days ago | 39 comments
I've a new working version of arc2c. It will be pushed on the git in the next few hours. It now includes a garbage collector and cons cells. Functions cons, car, cdr and type were added. The list function is not implemented yet (it supposes varargs functions), and quoting a cons cell (or quasiquoting anything) is still impossible. pr and prn work on cons cells too, so that you can display a list.

(As for GC : I had implemented my own, but it is bugged, inefficient and making the code hard to work with. So I removed it in favor of Boehm GC. For those interested in my attempts, I'll join a file named "fib.c", generated with that home-made GC.)

With the above compiler, I was able to implement a lot of basic stuff : no, len, map, rev, ... In other words, a part of the core functions. That's not a surprise, but I'm happy to see this is working, at least in the basic cases.

Fixnums are now pointers coerced into long ints (and not regular ints anymore) so as to make it work on a maximum of architectures. I haven't include absz's idea (http://arclanguage.org/item?id=5674) yet.

Oh, there are still a few bugs even in the small subset implemented yet I guess, so don't use arc2c in production code :)



2 points by almkglor 5836 days ago | link

Idea: define-for-syntax

Looking through how Chicken compiler works, it seems that Chicken detects syntax-case and friends, performing them at compile-time, However if a macro needs to use a function, the function should be defined using define-for-syntax. Perhaps our rules for macros:

1. Should be defined using 'mac

2. Any form that has a 'mac will be executed at compile-time, with the 'mac transformed to a macro definition:

  (let internal-fun
       (fn (x) `(foo ,x))
    (mac my-macro (z)
      (internal-fun z)))
3. Any functions that need to be used directly by macros must be added either in a 'let form like the above, or defined using 'def-syn.

Idea: Library

We probably need some sort of library system, like the part which handles 'ccc: add it only if it's needed, otherwise leave it out.

Hmm.

Idea: optional and varargs

It seems to me that optional variables might be implementable as varargs forms. Not sure though.

  (fn (foo (o niaw nil) (o grr niaw))
   (body foo niaw grr))

  =>

  (fn (foo . gs42)
    (with (niaw (if gs42 (car gs42) nil)
           gs42 (cdr gs42))
      (with (grr (if gs42 (car gs42) niaw)
             gs42 (cdr gs42))
        (body foo niaw grr))))
So all we need is to implement var-args, and special-case optional variables as above.

-----

2 points by binx 5836 days ago | link

An extra hash argument can implement both optional and named args. Representing optional args as varargs would complicate the source transformation when they are used simultaneously in the same function.

-----

2 points by almkglor 5836 days ago | link

Not really:

  (fn (niaw (o hmm) . rest)
    (body niaw hmm rest))

  =>

  (fn (niaw . gs42)
    (with (hmm (if gs42 (car gs42) nil)
           gs42 (cdr gs42))
      (let rest gs42
        (body niaw hmm rest))))
It's not at all complicated: just make the last argument take the value of your temporary rest argument. Edit: This is what I did in creating the p-m macro, and it works.

I'm not sure how adding an extra hash would work well if I might pass several different functions, with different optarg names:

  (set p1
    (fn (hmm (o niaw))
      (body hmm niaw)))
  (set p2
    (fn (hmm (o arf))
      (body hmm arf)))
  (set p3
    (fn rest
      (body rest)))
  ((if (something)
       p1
     (something-else)
       p2
       p3)
  1 42)
Edit: My approach appears to be somewhat similar to what pg did in the Scheme version - cref ac-fn, ac-complex-fn, ac-complex-args, ac-complex-opt in ac.scm

Since code is spec (grumble) I think it might be better to follow mostly what pg did, although I dunno, not sure ^^. How can passing in a hash table seamlessly emulate the same functionality while improving performance?

-----

1 point by binx 5836 days ago | link

The define-for-syntax form is just a sub-feature of CL's eval-when, why don't us give a full eval-when support?

-----

1 point by almkglor 5836 days ago | link

Hmm. I'll think about that.

eval-when would be used, I think, only to differentiate between compile-time and run-time. It's not a part yet of ArcN. Hmm. Lemme think more deeply about this.

Edit: This seems like a good idea ^^

-----

2 points by almkglor 5837 days ago | link

Regarding 'if: We could insert a code walker which transforms 4 or more arg forms of 'if to conventional 'if:

  (if
    (hmm)
      (niaw)
    (grr)
      (woof))

  =>

  (if (hmm)
      (niaw)
      (if (grr)
          (woof)))
Also, we can use Anarki-specific 'ssexpand and 'ssyntax to transform symbol syntax, probably by inserting another step in the compile process (which is why I suggested structuring arc2c in: http://arclanguage.com/item?id=5598 ); However ssyntax which transforms to ((compose a b) ...) has to be transformed to (a (b ...)) because of macros, such as p-m:def.

-----

1 point by sacado 5837 days ago | link

Excellent idea :)

-----

4 points by almkglor 5836 days ago | link

wrote a code walker, on the git.

Basically to use:

  (def your-function (argument-data)
    (code-walk a-variable argument-data
      (if
        (something-you-need-to-process a-variable)
          `(stuff ,(car a-variable) ,(cdr a-variable))
        ; else return the variable as-is
          a-variable)))
code-walk will automatically skip ' as well as the constant parts of `(), but will reprocess when it sees a comma or comma-at `( ,...)

See to-3-if sample

Also I removed the err in 'source - since part of codegen now emits the code in list form rather than AST, source just passes it if it's not an AST

Edit: also added ssyntax support

-----

1 point by almkglor 5836 days ago | link

Closures:

It seems that closures aren't nested (i.e. do not refer to other frames). In short, when we have a closure within a closure:

  (fn (x)
    (fn (y)
      (fn (z)
        (prn x)
        (prn y)
        (prn z))))
This compiles to (pseudo-Arc):

  (fn (self x)
    (make-closure
      (fn (self y)
        (make-closure
          (fn (self z)
            (prn self.0)
            (prn self.1)
            (prn z))
          (list self.0 y)))
      (list x)))
(here, self means the closure of the function, not 'afn self).

Because the frames aren't shared, this means the current closure-conversion model can't be easily(?) modified made to work on the following code:

  (fn (x)
    (list
      (fn () x)
      (fn (v) (set x v))))
I'm not sure how to resolve this unfortunately. And again I'm not 100% sure of my analysis.

-----

4 points by stefano 5834 days ago | link

You could try transforming

(fn (x) (list (fn () x) (fn (v) (set x v))))

to

  (fn (x)
    (let x (cons x nil)
      (list
        (fn () (car x))
        (fn (v) (set-car x v)))))
This way the adress of the cons cell x isn't shared, but the cell itself is shared. The compiler would do this transformation automatically.

-----

3 points by almkglor 5834 days ago | link

Done and on the git.

Some notes:

1) I use a new structure, the sharedvar, which doesn't correspond to any Arc structure. See the other post which had me blinking stupidly at stefano's transformation before I realized how cool it was: http://arclanguage.com/item?id=5784

2) This new structure is untyped (i.e doesn't have type tags, unlike the pair and symbol structures). I intend to make shared variables "seamless" to the upper Arc though, so this should be fine...

3) My original name for this structure was closure-settable, which I shortened to sharedvar instead.

PS f34r my new gravatar

-----

1 point by stefano 5834 days ago | link

Wow. You implemented it very quickly.

Making the new structure untyped shouldn't create problems, but it could make development difficult: if you have a bug in the transformer or in something related to it, you'll probably get a Seg. fault error instead of a clean "Not a sharedvar" error. The check could then be removed when the code is released.

-----

1 point by almkglor 5834 days ago | link

Hmm. So far the transformation seems correct anyway; and really, the transform is quite simple and appears mathematically correct. Also, because of the way the compile-file driver is structured, it would be possible to remove/replace each individual transform.

Which reminds me, we need to have a proper error continuation too.

-----

2 points by almkglor 5834 days ago | link

Huh.

blinks stupidly

Cool.

Why didn't I....

Cool.

Blink

Cool.

Edit: Okay, I managed to snap out of shock.

The base idea is pretty good, although the 'cons cell (which is composed of a type id, a car pointer, and a cdr pointer) can be replaced with a smaller "closure-settable" cell, which would be an untyped object composed of a single pointer:

  (fn (x)
    ; % means it's a primitive
    (let x (%closure-settable x)
      (list
        (fn () (%closure-settable-read x))
        (fn (v) (%closure-settable-write x v)))))
Because the closure variable abstraction is not accessible from the high-level Arc language, the closure-settable doesn't need to be typed tagged

-----

3 points by almkglor 5835 days ago | link

Variadic functions now work:

  (set list
    (fn rest
      rest))

  (prn (list 9 8 7 6 5 4 3 2 1))

-----

2 points by sacado 5834 days ago | link

wow, congratulations, you are much more productive than I am ! I won't work a lot on the compiler this week as I am far from my own computer, so I guess many things will have changed next time I'll explore the code :)

-----

2 points by almkglor 5834 days ago | link

Thanks ^^. I'm not that productive, really - I haven't figured out yet how to properly handle environments like so:

  (let private-variable nil
    (set set-private
      (fn (x) (set private-variable x)))
    (set get-private
      (fn () private-variable)))
I've been circling this problem for some time now, getting nowhere quick.

The problem is that the current style of implementing closures - where local and closure variables are copied to the closure structure - strikes me as a premature optimization. A useful one - if closures are not nested, then all variable references are O(1). However for shared mutable variables we need to use a different strategy. I'm trying to figure out some way of retaining the current closure style for cases where variables are not mutated, versus for cases that closure variables are.

BTW how come no one else seems to want to work on the compiler? It seems that aside from you I'm the only commiter?

-----

1 point by absz 5834 days ago | link

Well, I'm watching with fascination, but it's over my head. I'm working through Essentials of Programming Languages (Friedman, Wand, and Haynes), though, so maybe soon ;)

-----

3 points by wfarr 5837 days ago | link

Because the scope of what you're doing is fairly large in its own, I'd really recommend making your own repo on hub just for arc2c - such that anarki "core" doesn't have any adverse side-effects.

-----

3 points by sacado 5837 days ago | link

That's an excellent idea. I've just started a git at http://github.com/sacado/arc2c . The new version is published on it. If I configured it correctly, anyone can push modifications on it. I have 2 invitations to github left, by the way.

-----

2 points by almkglor 5837 days ago | link

  ERROR: Write permission to sacado/arc2c denied to AmkG.
  fatal: The remote end hung up unexpectedly
  error: failed to push to 'git@github.com:sacado/arc2c.git'

-----

1 point by sacado 5837 days ago | link

Hmm... Thaks for the information. I can't see how I can say "this git is open to everyone" in the config menus. Any idea ?

-----

1 point by almkglor 5837 days ago | link

Hmm. Can't seem to do this either. Erk.

I suppose nex3 specifically requested github for special treatment of Anarki? ^^

-----

2 points by sacado 5837 days ago | link

Anyway, I can add specifically anyone who asks me to. You are now an authorized "committer", btw.

-----

2 points by almkglor 5837 days ago | link

Thanks, I'll push something in maybe an hour or two, am at the office.

The push I was going to make was simply to wrap some of the private functions in codegen.arc in a 'let form.

-----

1 point by skenney26 5837 days ago | link

I'd appreciate an invite if you still have any left.

-----

1 point by sacado 5837 days ago | link

done.

-----

1 point by skrishna23 5837 days ago | link

can I have one please ? TIA.

-----

1 point by almkglor 5837 days ago | link

what's your email address?

-----

1 point by skrishna23 5837 days ago | link

my user id at yahoo.com

-----

1 point by almkglor 5837 days ago | link

Sent!

-----

2 points by almkglor 5837 days ago | link

Incidentally, arc already has a "union" function. I think the arc2c version has the following relationship:

  (arc2c!union a b) == (arc!union is a b)
Not sure though.

-----

1 point by sacado 5837 days ago | link

Ok, I'll rename arc2c's union to something else or I'll remove it and use (union is a b).

-----

3 points by slashcom 5837 days ago | link

I was going to play around with it, but I it wouldn't load because debut.arc isn't in the repository.

-----

1 point by sacado 5837 days ago | link

Sorry about that. I'll correct it this evening (GMT+1). As for now, you can still find debut.arc in Anarki. It didn't change since last time.

-----

2 points by almkglor 5837 days ago | link

I can't find debut.arc in Anarki either.

Some more Anarki-related questions:

1. Do you intend to use Anarki-specific features? This is so that I know if I can use Anarki in any mods.

2. Do you intend to still use the arc2c copy on Anarki, say as a stable branch?

-----

1 point by sacado 5837 days ago | link

Yep. Just checked that. In fact, "debut.arc" was renamed "structs.arc". Just change this in "arc2c.arc", and it will be working.

1. Yes, I will probably. I only work with the Anarki version, so when I'm finding something useful in the source, I don't even know if it's "official" or not.

2. Yes. Using Anarki as a repository for the stable versions of arc2c seems a good idea.

-----

1 point by almkglor 5837 days ago | link

Also, another question - Anarki has a CONVENTIONS file, will we be following this too? One Anarki convention that isn't followed is with regards to tabs and indentation, is it OK to change the tabs to follow Anarki conventions or do you prefer the current indentation?

-----

1 point by sacado 5837 days ago | link

Oh, sure, I never really checked the CONVENTIONS file, but I think it's better to indent this way (it's the tradition after all:). I usually don't do it naturally because it's not the way I learnt it, but I guess I should follow the conventions anyway. So, OK, let's change the tabs.

-----