Arc Forumnew | comments | leaders | submitlogin
A (possibly not very important) bug in 'annotate.
4 points by almkglor 5922 days ago | 24 comments

  arc> (= f (fn () nil))
  #<procedure: f>
  arc> (= z (annotate 'mac f))
  #3(tagged mac #<procedure: f>)
  arc> (z)
  nil
  arc> (= y (annotate 'fn z))
  #3(tagged fn #3(tagged mac #<procedure: f>))
  arc> (y)
  nil
  arc> (= x (annotate 'mac y))
  #3(tagged mac #3(tagged fn #3(tagged mac #<procedure: f>)))
  arc> (x)
  Error: "apply: expects type <procedure> as 1st argument, given: #3(tagged fn #3(tagged mac #<procedure: f>)); other arguments were: ()"
The problem is this:

  (define (ar-tag type rep)
    (cond ((eqv? (ar-type rep) type) rep)
          (#t (vector 'tagged type rep))))
I think it should be:

  (define (ar-tag type rep)
    (cond ((eqv? (ar-type rep) type) rep)
          ((ar-tagged? rep) (vector-set! rep 1 type) rep)
          (#t (vector 'tagged type rep))))
The reason I'm concerned about this, is because I intend to extend tagged objects with a fourth entry, a hash of "attached" key-value pairs, so that an object may have a definable setterfunction.


7 points by nex3 5922 days ago | link

I disagree heartily. The whole point of annotate is to be able to assign a type to an arbitrary value. That breaks if it misbehaves for tagged values and resets the tag instead of adding a new one. In particular, it seems wrong that right now you could trivially write retag ([annotate _1 (rep _2)]), whereas with this change, it would be impossible to write annotate as it behaves now.

One thing I'm reasonably sure of is that this isn't a bug - this is how PG intended annotate (née tag), type, and rep to work. See http://www.paulgraham.com/ilc03.html, for example.

I also dislike, somewhat less vehemently, the idea of adding Python-style object hashes. Especially if you're going to modify the core axioms to do it, when it would be perfectly possible to do it in-Arc using annotate and friends.

-----

2 points by almkglor 5921 days ago | link

Apparently then the bug is that Arc doesn't really support annotate very well yet:

  arc> (= f (annotate 'x 1))
  #3(tagged x 1)
  arc> (+ f 1)
  Error: "+: expects type <number> as 1st argument, given: #3(tagged x 1); other arguments were: 1"
That said, it seems to work with very deeply nested (annotate 'foo (annotate 'bar (annotate 'nitz (fn () (prn "I'm here!"))))).

Re the "in-Arc" solution - the problem is trivially attaching a set of properties to an object. For example, an object might have an "assign" aspect, might have a "length" aspect, might have a "keys" aspect, might have "writeable-keys", etc. It's possible to do that using (annotate `(= ,setterfunction) readerfunction) or something, true, but this requires us to search through a possibly deeply nested annotate.

That said the add-attachment and get-attachment stuff (from lib/settable-fn.arc) can be rewritten to use annotate instead:

  (def add-attachment (k v s)
    (annotate `(,k ,v) s))
  (def get-attachment (k s)
    (if (is s (rep s)) ; is this relationship assured?
        nil
        (let ty (type s)
            (if (and (acons ty) (caris ty k))
                (cadr ty)
                (get-attachment k (rep s))))))
However as pointed out above, Arc does not seem to always destructure tagged objects.

-----

3 points by nex3 5921 days ago | link

Yeah, I see the issue... what would be ideal would be being able to do something like

  (def add-attachment (k v s)
    (annotate 'settable-fn
      (list k v s)))
and then somehow define what it means to have a "settable-fn" in a functional position.

-----

1 point by absz 5921 days ago | link

That should be doable with a new primitive and a modification to ar-apply in ac.scm. If we define a primitive called, say, behave (or rather, a much better name that I can't think of), such that

  (behave
    'foo
    (fn (self x y) (list (+ (self 0) x) (- (self 1) y)))
lets us write

  (let obj (annotate 'foo '(42 2))
    (rep (obj 5 2)))
to return (47 0), and we modify ar-apply to look for that, this should work. behave should be something like

  (define (ar-behave type fn)
    (hash-table-put! ar-behaviours type fn))
, and ar-apply would change

  ((ar-tagged? fn) (ar-apply (ar-rep fn) args))
to

  ((ar-tagged? fn) (ar-apply (hash-table-get ar-behaviours (ar-type fn)) args))
. (This is all untested.) In fact, we could even move everything but functions (and macros?) out of ar-apply and define them all in arc.arc, removing the (ar-tagged? fn) check, but this would break on the release of a new arcn.tar.

(Oh, and behave is still an awful name.)

-----

2 points by nex3 5921 days ago | link

I'm working on something like this. Should be done in half an hour or so.

-----

2 points by nex3 5921 days ago | link

It took longer than I expected, but http://www.arclanguage.org/item?id=3743.

-----

2 points by almkglor 5921 days ago | link

Here's an objection:

The main reason my settable-fn works the way it does is so that attachments are orthogonal to annotations. An object might be tagged, or it might be attached, or it might be both tagged and attached.

This means that I could have defined a function, tagged it as 'table, and provided a '= attachment for a setterfunction and a 'keys attachment for a redef'ed 'maptable and 'keys. This way, I don't have to modify, say, 'each, which will simply see that it's a 'table and pass it to 'maptable, without ever realizing that it isn't a hash table.

With this system, everything that takes an attachment must be a settable-fn (or whatever). This means that I need to modify about a dozen existing macros so that it will work with my "table", so that they will also check for a 'keys attachment, and probably I want to do this for any new macros. Sure, I could redef 'type, but this gets more complicated (and potentially risky).

So no, I don't agree with this mod, because I think attachments should be orthogonal to annotations. I don't want an object-with-attachments to have its own annotation, I want the user to specify his or her own annotation for the object. In any case, since the mod has been pushed on arc-wiki, I'll have to work around it, possibly having to redef 'type.

-----

2 points by nex3 5921 days ago | link

I'm not sure I understand... defcall doesn't really have much at all to do with annotate. It certainly doesn't change the way it works. It just allows you to make user-defined types work like functions.

In any case, it seems un-Lispy to me in general to redefine behavior by attaching things to objects. I'd rather see us redefining and tweaking the verbs rather than adding information to the nouns. This is also, I think, how PG envisioned annotate and friends working. From http://www.paulgraham.com/ilc03.html :

"If you want to overload existing operators to do the right thing when given your new type, you don't need anything new in the core. As long as you have lexical scope, you can just wrap a new definition of the operator around the old one. So if you want to modify print to display objects of your new type foo in a special way, you write something like this:

  (let orig print
    (def print (x str)
      (if (is (type x) 'foo)
          ...new code...
          (orig x str))))"
I don't know, maybe I'm being short-sighted and attaching information to objects is really necessary. But I'm not seeing a case right now where it wouldn't be just as easy (or easier, with stuff like defcall) to define something like an attached-object type and use that.

-----

1 point by almkglor 5920 days ago | link

defcall specifies how an 'annotate 'd object will work in function position. This meant that it dispatches on the claimed type of the object, not on the real type. So (at least in your original version) a type masquerading as another type will be difficult to implement:

  arc> (= test (annotate 'table (fn (x) x)))
  #3(tagged table <procedure>)
  arc> test!x
  Error: "Can't get reference"
I've since modified it so that if it's a tagged type, 'ref will perform apply on its representation. This means that currently, (= call 'type ref) will cause an object typed as 'type to dispatch as if it were its representation, while using (defcall 'type ...) will dispatch based on its type (meaning it can't be faked)

-----

3 points by nex3 5920 days ago | link

I don't think types should masquerade as other types, by which I mean they shouldn't annotate themselves with other types' symbols. I think the proper way to act like another type is to behave like that type, not to annotate yourself with that type.

The thing is, if we expect (annotate 'table (fn (x) x)) to act just like a table out of the box, we have a lot of work to do. Every table axiom has to check for an annotation and recurse if one exists.

This may not seem so bad for tables, but consider: if ((annotate 'table (fn (x) x)) 'foo) works, shouldn't ((annotate 'cons (fn (x) x)) 1)? What about (+ (annotate 'num (fn (x) x)) 2)? What does that even mean?

It seems to me that the easiest and most consistent way to mimic other types is to annotate with a new type but to redef functions like keys and defcall to work in functional position.

I urge you to check out settable-fn2.arc if you haven't already - I re-implement get- and add-attachment using this style of annotation, and it comes out quite nicely. Rather than annotating the attached functions with their types, I add a 'type attachment which overrides isa. It appears to work fine with file-table.arc, too.

-----

2 points by almkglor 5920 days ago | link

From my point of view, attachments should be orthogonal to types. Basically, an attachment is any piece of information you want to attach to an object, and lives and dies with that object. That an attachment is used to overload 'keys or '= is just a use of the attachment concept.

For example, we might want to build a reader which keeps track of line numbers. The reader's output is still 'cons and 'sym, etc., but with an attachment. Each 'cons cell has a 'linenumber attachment which we can use. For example, a macro whose syntax has been violated would be able to report the line number where this violation occurs. This is useful if the macro is used often enough and there is a need to locate the line number of the error, or if its syntax is like CL 'loop and you expect it to span several lines.

In all cases, the cons cell produced by this hypothetical reader is a cons cell. Its representation is a cons cell and is only a cons cell. However, we can extract additional data from it. After it passes through the evaluator and is discarded as trash, its attached data can be thrown away.

In any case file-table.arc only cares that settable functions work, and settable functions only care that attachments work. Whether we make attachments orthogonal to types, or separate stuff-with-attachments as types may not really matter so much anyway. This is Lisp, after all.

-----

2 points by nex3 5920 days ago | link

I agree that attachments should be orthogonal to types - that's why I added the "isa" overloading to settable-fn2. But I don't think annotations should be orthogonal.

The thing is, there's no way we'll be able to add arbitrary attachments to any object and have it continue to behave just as if there were no attachments in Arc. We'd need to either modify the core to give each object a Python/Ruby/Javascript/etc-style implicit table, which I don't think PG is likely to be very fond of (and which I don't think is a good idea besides); or we need to accept that there will be some cases where we won't be able to get attachments without a few compromises.

-----

3 points by almkglor 5920 days ago | link

Well, I thought annotations were types!

Anyway:

"Now this is the noble truth of the origin of suffering: it is this attachment which leads to renewed existence, accompanied by delight and lust, seeking delight here and there, that is, attachment to sensual pleasures, attachment to existence, attachment to extermination."

Therefore... buddha pg, please enlighten us and deliver us from attachment! ^^

-----

3 points by nex3 5920 days ago | link

Buddha PG will free us from suffering by means of s-expression Koans and well-timed whacks to the head!

-----

1 point by nex3 5921 days ago | link

Also, I just pushed a settable-fn2.arc that uses annotations how I envisioned them being used (it's also pure Arc). Hopefully that should make it more clear what I'm thinking.

-----

1 point by almkglor 5921 days ago | link

The problem is the destructuring. In my proposed solution already we might have some pretty deep attachment layers:

  (def add-attachments (atts obj)
    (if (no atts)
      obj
      (add-attachments (cdr:cdr atts) (add-attachment (car atts) (car:cdr atts) obj))))
  (= foo
     (with (...)
       ...
       (add-attachments
         (list 'keys key-getter-function
               '=    setter-function
               'len  len:key-getter-function)
         reader-function)))
In such a case, annotate ends up being effectively a cons (with annotate==cons, type==car, rep==cdr). The structure then approaches an association list. O(n) lookups and all that.

When the road is clean, more people can explore where it leads.

-----

3 points by nex3 5921 days ago | link

The thing is, I don't think annotate should be used as cons. The idea is that it attaches symbols which represent types - not that it bolts on new data. If you want data, you should annotate a cons or a table (or a cons with a table). Like, for example, if for some reason you wanted to create a point type with annotate, I think you wouldn't want to do

  (annotate x y)
but rather

  (annotate 'point (cons x y))
That's a silly example, because points are so simple, but for your attachment functions, I think adding to an actual list or table is a better solution than treating annotations as cons cells.

-----

1 point by almkglor 5921 days ago | link

Ah, the code example you gave threw me. I suppose what you really meant was something more like this:

  (def add-attachment (k v s)
    (if (isa s 'settable-fn)
        (do (= ((car (rep s)) k) v)
            s)
        (annotate 'settable-fn
            (cons (fill-table (table) (list k v)) s))))
re annotations as cons cells - well, that's already how they work, and apparently it's not a bug, as you mentioned.

-----

4 points by nex3 5921 days ago | link

Oh, I certainly don't mean that annotations aren't equivalent to cons cells. That's definitely true. I'm just saying that that doesn't mean we should use them like we use cons cells. I think their primary use should be the one your code example shows: to add a type to some other object.

-----

1 point by offby1 5922 days ago | link

I guess I'm totally confused about "annotate", then; I thought that (= y (annotate 'fn z)) would return #3(tagged mac #<procedure: f>) -- i.e., it would _replace_ the existing annotation, rather than _adding_ to it.

-----

4 points by absz 5922 days ago | link

No; as nex3 observed, it's supposed to add it (cf. http://www.paulgraham.com/ilc03.html). Why? This is more general. Right now, we can define reptag to do what you want:

  (def reptag (typ obj)
    (annotate typ (rep obj)))
If we just had reptag, we couldn't define annotate.

Also, annotate obeys two useful identities:

  (type (annotate x y)) --> x
  (rep  (annotate x y)) --> y
However, reptag does not:

  def                      --> #3(tagged mac #<procedure>)
  (rep (annotate 'fn def)) --> #3(tagged mac #<procedure>)
  (rep (reptag   'fn def)) --> #<procedure>
Because of the type-replacing behaviour, that identity does not hold for tagged objects. I consider that a strike against it as well.

-----

2 points by almkglor 5921 days ago | link

Hmm. I suppose the real issue here is what pg intends to happen when annotating already tagged objects:

  (= y (annotate 'bar 'hmm))
  (annotate 'foo y)
3 options:

  ; option1: current Arc
  arc> (annotate 'foo y)
  #3(tagged foo #3(tagged bar hmm))
  arc> y
  #3(tagged bar hmm)
  ;option2: my proposal
  arc> (annotate 'foo y)
  #3(tagged foo hmm)
  arc> y
  #3(tagged foo hmm)
  ;option3: something else entirely
  arc> (annotate 'foo y)
  #3(tagged foo hmm)
  arc> y
  #3(tagged bar hmm)

-----

4 points by nex3 5921 days ago | link

I think it's pretty clear that he intends option 1, as that's how it actually works, and as absz pointed out, is a strict superset of the functionality of option 2 (and has nicer properties, too).

-----

1 point by cooldude127 5922 days ago | link

i agree, this seems like a bug.

-----