Arc Forumnew | comments | leaders | submitlogin
Department of obscure errors: quasiquote fails with dotted lists
6 points by conanite 5598 days ago | 10 comments
I spotted this today

  arc> (with (a 1 b 2) `(foo (,a . ,b)))
  (foo (1 . 2))
  arc> (let foo 'bar `(,foo (a . b)))
  Error: "map: expects type <proper list> as 2nd argument, given: (a . b); other arguments were: #<procedure:...w/src/arc/ac.scm:247:14>"
It might look unimportant, but it's impossible to use a macro to generate a macro that takes a rest param:

  arc> (mac foo-mac (bar)
    `(mac ,bar (baz . args) (prn ',bar " and " args)))
  Error: "map: expects type <proper list> as 2nd argument, given: (baz . args); other arguments were: #<procedure:...w/src/arc/ac.scm:247:14>"
If this is worth fixing, I guess it's in ac-qq1 in ac.scm:

  (define (ac-qq1 level x env)
    (cond 
          ; ...
          ; lots of conditions
          ; ...

          ((pair? x)
           (map (lambda (x) (ac-qq1 level x env)) x)) ; here's 'map
          (#t x)))
Except if the fault is here, I don't understand why the first example above works ( `(foo (,a . ,b)) )


2 points by palsecam 5597 days ago | link

I'm not sure if this correctly solves the right problem and without breaking other things, and I'm nearly certain there is a better solution anyway, but here is a pseudo-patch:

In ac.scm, add:

   + (define (map1-dotted func xs)
   +  "Limited 'map but it can handle a dotted list. Needed for 'ac-qq1."
   +   (cond ((pair? xs) 
   +          (cons (func (car xs)) (map1-dotted func (cdr xs))))
   +         ((not (null? xs)) (func xs))
   +         (#t xs)))  ; xs is null
In the definition of 'ac-qq1, change:

     [... conditions ...]
     ((pair? x)
   !  (map (lambda (x) (ac-qq1 level x env)) x))
by:

     [... conditions ...]
     ((pair? x)
   !  (map1-dotted (lambda (x) (ac-qq1 level x env)) x))
Result:

   arc> `(a . b)
   (a . b)
   arc> (with (a 1 b 2) `(foo (,a . ,b)))
   (foo (1 . 2))
   arc> (let foo 'bar `(,foo (a . b)))
   (bar (a . b))
I tried in Common Lisp and 'map doesn't work either with a dotted list. It seems like using 'map here, in 'ac-qq1, is attractive (simple) but not totally correct.

-----

3 points by rntz 5597 days ago | link

The reason why the first example works is that `(foo (,a . ,b)) is syntax for (quasiquote (foo ((unquote a) unquote b))), which doesn't contain dotted lists.

I'll try my hand at fixing this when I have the time.

-----

2 points by rntz 5594 days ago | link

The fix I'm using for the time being is this:

    diff --git a/ac.scm b/ac.scm
    index 3304953..5263d28 100644
    --- a/ac.scm
    +++ b/ac.scm
    @@ -238,7 +238,7 @@
             ((and (pair? x) (eqv? (car x) 'quasiquote))
              (list 'quasiquote (ac-qq1 (+ level 1) (cadr x) env)))
             ((pair? x)
    -         (map (lambda (x) (ac-qq1 level x env)) x))
    +         (cons (ac-qq1 level (car x) env) (ac-qq1 level (cdr x) env)))
             (#t x)))

     ; (if) -> nil
This does weird things to quasiquote in medial position (see below), but IMO doing that is suspect, and it fixes both the "can't use dotted lists in quasiquoted expressions" bug and the "dotted unquote only works sometimes" bug:

    arc> `(foo . bar)
    (foo . bar)
    arc> `(foo . ,(join '(x) '(y)))
    (foo x y)
For those interested, more detail follows on the corner cases of quasiquotation in the underlying mzscheme. As it turns out, mzscheme (possibly schemes in general) has really weird quasiquoting rules. Consider the following:

    ; we begin with normal examples
    > (define x 'X)
    > (define X 'value)
    > `(a ,x)
    (a X)
    > `(a `(b ,,x))
    (a (quasiquote (b (unquote X))))
    
    ; quasiquoting in dotted position has interesting results:
    > `(a . `(b ,x))
    (a quasiquote (b (unquote x)))
    > `(a . `(b ,,x))
    (a quasiquote (b (unquote X)))
    > (eval `(list . `(b ,,x)))
    quasiquote: bad syntax in: quasiquote
    
    ; unquoting in terminal position results in splicing, as expected
    > '`(a . ,x)
    (quasiquote (a unquote x))
    > `(a . ,x)
    (a . X)
    > `(a . ,(list x))
    (a X)
    ; unquotes in medial positions error, which is good:
    > `(a unquote x X)
    stdin::1874: unquote: expects exactly one expression at: (#<syntax::1879> #<syntax::1887> #<syntax::1889>) in: (quasiquote (a unquote x X))
    
    ; however, the same is not true of quasiquotes in medial position.
    > `(a quasiquote b c)
    (a quasiquote b c)
    ; in fact, quasiquotes in medial position require increased unquoting of
    ; subsequent elements - quasiquoting in dotted position, above, is a special
    ; case of this
    > `(a quasiquote ,x ,x)
    (a quasiquote (unquote x) (unquote x))
    > `(a quasiquote ,,x)
    (a quasiquote (unquote X))
    ; however, if the list following quasiquote is dotted, this does not happen...
    > `(a quasiquote ,x . y)
    (a quasiquote X . y)
    ; ... UNLESS the dot comes immediately after the quasiquote
    > `(a quasiquote . ,,x)
    (a quasiquote unquote X)
Since arc quasiquotation compiles down into scheme quasiquotation, I'm unsure of the best way to handle all this at the arc level. What should all of these corner cases mean in arc? Moreover, consider that all uses of medial quasiquote (eg '(list quasiquote 2)) result in syntax errors when evaluated in scheme, but in arc, evaluating eg '(list quasiquote 2) is not a syntax error, and will depend upon the value of 'quasiquote; in fact, the same is true of most special forms:

    arc> (let quasiquote 0 (list quasiquote 1))
    (0 1)
    arc> (let assign 0 (list assign 1))
    (0 1)
Intuitively, binding values to special forms seems highly suspect, but there you have it.

-----

1 point by conanite 5597 days ago | link

Aha, I am enlightened, that explains why 'map doesn't complain.

It's still a mystery (to me) how ac manages to substitute unquote b because it's not a proper unquote. At some point, the x in

  (map (lambda (x) (ac-qq1 level x env)) x))
will be bound to the symbol 'unquote, so the compiler should never see (unquote b) ...

clearly, the only logical explanation is magic :)

-----

2 points by rntz 5597 days ago | link

The answer is that arc quasiquotation gets compiles down to scheme quasiquotation, and mzscheme's quasiquote handler understands (... unquote bar) to be equivalent to (... (unquote-splicing bar)). Since the arc compiler doesn't understand unquotes of this style, you'll only get the correct results when the expression unquoted is the same in arc and scheme - ie: it's either a literal, or it's a local variable. For example:

    ; this works because the local 'x in arc is compiled to 'x in scheme as well
    arc> (let x 0 `(a . ,x))
    (a . 0)
    ; this works because '+ in scheme adds numbers just as in arc
    arc> (let x 0 `(a . ,(+ x 1)))
    (a . 1)
    ; the following examples do not work
    arc> (let x '(b) `(a . ,(join x '(c))))
    Error: "reference to undefined identifier: join"
    arc> (let x '(b) `(a . ,(+ x '(c))))
    Error: "+: expects type <number> as 1st argument, given: (b . nil); other arguments were: (c)"
    arc> (= x '(b))
    (b)
    arc> `(a . ,x)
    Error: "reference to undefined identifier: x"
It's funny how a surface bug turns out to lead to something deeper in this way. I feel like Alice down the rabbit hole.

-----

2 points by fallintothis 5597 days ago | link

I also noticed this error back when I ported quasiquote from GNU clisp (http://arclanguage.org/item?id=9962). I was happy to find that the port fixed this bug when I similarly made a macro-defining macro where the newly-defined one took a rest parameter. Small world.

-----

1 point by conanite 5597 days ago | link

I was also happy to find rainbow doesn't have this issue either ... but I'm putting that down to luck :))

small world indeed

-----

2 points by CatDancer 5598 days ago | link

This looks like a shorter example...:

  arc> `(a . b)
  Error: "map: expects type <proper list> as 2nd argument, given: (a . b); other arguments were: #<procedure:/tmp/arc/ac.scm:241:14>"

-----

1 point by conanite 5598 days ago | link

I used slightly longer examples to show the difference when the list members were unquoted. I don't understand at all why it works, because as far as I can tell, the same code branch (that maps over the quasiquoted list) is invoked in both cases.

-----

3 points by pg 5590 days ago | link

Robert fixed it in 3.1.

-----