The conflict I see is between pg releasing Arc at this very early stage because he found it useable vs you all going batsh*t over the volatility. You are both right! It is great that the users are chomping at the bit for something stable, but pg warned everyone that this was an experimental release and he planned to trash all our code at will. :)
Looking at Arc as a Lisp veteran I can assure you all that you totally need to reset your expectations and sign up for a fun exploration of a possible (emphasis on possible, because the more I learn of Lisp-1 the more I consider it a grave error) ... of a possible sweet spot in the abstract Lisp language space.
If Lisp-1 does turn out to be a grave error, it doesn't seem like it would be that difficult to either add the Scheme features that make Lisp-1 work, or turn it into a Lisp-2, given the side of the Arc codebase.
Right now, I'm just looking for the best Lisp to develop web applications with. Ruby on Rails is my benchmark, and I think it can be improved on with Lisp, but that remains conjecture at this point.
Just to be clear. I'm not asking for stability at this point. I agree that maintaining backward compatibility would waste time & effort. Arc can still evolve like crazy and break existing code, but it would be nice to have a way for the community to feed patches & bug fixes to pg besides the Arc forum.
"one of my frustrations ... is dealing with the myriad of implementation choices"
As Sartre said, you are not free to be not free. Have fun with Arc, use Common Lisp. Lisp-1 buys you nothing but aggravation, minimal just means you end up building your own full tool chest that is incompatible with anyone else's so you can never share code.
Having ported my groovy Cells project to Arc, I can confirm that it was a pain where I sit that "only the names have been changed" in several cases. :) But pg gets the benefit of the doubt from me because I own both his books and he kinda rocks when it comes to Lisp.
The funny thing is that perhaps unbeknownst to you all the Lisp establishment is ragging on pg for Arc being too much like Lisp, you (and I a little) are hosed that he diverged from Lisp, so him being smart I would guess he is now ignoring everyone. :)
The only thing I can offer is that it totally sucks to have an installed base and not be able to change things cuz you will break all the code ofyour users. Only at times like this is an inventor free to change everything and anything, I guess pg did.
No, it does not work that way, although people who do not understand macros (such as Guido) live in fear of that hobgoblin.
Macros are not used to create unrecognizable languages. They are used when an API has grown to the point where writing the code to use it can be automated. That is probably hard to parse if you fear macros, because you can only fear macros if you do not know how they are used. But the idea is simple.
This little call tends to require this little call before it and this little call after it, or something like that. And this pattern appears often enough, or the Lisp developer recognizes it as the sort of thing that will appear again and again, and they just say, macro time!
They then give the macro a totally comprehensible name derived from the bits of the API being hit and, golly, no confusion.
The other time you see macros is in things like aif. There will only be so many of these, and they will confuse people not at all.
It seems to me some people want macros to be a problem. They never are.
Yes, well. In the office no one wants to touch my code, because it's built from bunches of macros, and no one else here knows how macros work. Oh well. Could be just me, I suppose.
Edit: gets even worse when I use C macros in C code ^^, they even instated a rule that loops should use for(i=0;i<limit;++i) and not my otherwise shorter repeat(i,limit) macro
We almost agree. :) I don't know, when they look at your functions do they know what they do? When they see a class name do they understand the class hierarchy? Or do they start browsing? As for repeat(i, limit) being banned, I presume because no one could guess what it does, well, I am looking at bartender's school. The more I learn about software the harder it is to work with some people. :) But I don't blame Dilbert on the C preprocessor.
LOL. I think the problem, partly, is the fact that we're attached to a Japanese company and the Japanese might not have that good a grasp of what "repeat" means (they tend to have a sneering attitude to anything non-Japanese, which means they suffer in the english-language department). I did manage to talk some guys into using repeat, but they got ordered by the Japanese to change it to "for", presumably because the Japanese knew "for", didn't know what "repeat" meant, and couldn't figure out how #define worked.
Edit: Too bad I'm a teetotaler, I'd have gone to bartender school too.
I saw him speak on Arc at ILC 2003. Paul is a legendary Lisper (he made money with it after 1990! (And wrote two great books (and has evangelized it terribly effectively))), He would be welcome anyway, Arc would certainly seal the deal. What am I saying...
If pg wants to go, he'll be there? I just hope he calls out Norvig for being a traitor and switching to Python. :) A bit of a story here:
I did not understand why there was a problem with find when nil appears in the searched list, especially in your example where it appeared last. Sorry if it is obvious, have not had my coffee yet. In your example, even if we move nil to the first position, isn't nil the first short list? It's length is zero. If you meant "nils don't count!", my question is still why not? because in my experience (13 years) they always do. But if you think this is a problem, then the only problem is that you did not say what you wanted in your test:
(find [< 0 (len _) 3)] lists)
If you need something more general to exclude nils:
(find [when _ (even (len _))] lists)
I will pause at this point to observe that you said your sample ifhave usage was "as concise as the existing Arc" but I missed the existing Arc, which might have helped me understand where you see a problem.
As for optional parameters, there you have a case! I almost never need it in CL, but one can do what in arc would be:
Oh. We had the same question in re hash tables where one might want to distinguish between a key stored with the value nil and a missing key, not possible with Arc since it does not store the key when the value is nil, and pg replied that not being able to distinguish the two was not an issue in practice. Perhaps the same "show us the use case" rule applies here?
As I said, there are plenty of half-baked solutions to this problem, and mem is an example; it distinguishes two possibilities by returning either a cons or nil. What I'm looking for is a general abstraction that obviates the need for such kludges.
You say "kludge", I (and apparently pg) say "you do not have a use case, this is an imagined problem" and one should not build languages around imagined problems, you end up with Java. I asked for a use case and you have not supplied one. I am not saying one does not exist, I am saying any discussion of a "fix" is meaningless until we have a target.
Consider again the very similar case of Arc tables being unable to differentiate "key not found" from "key associated with nil". I was halfway thru a brilliant retort to pg on this with an example from my own real live working code (where I used CL's ability to differentiate the cases) when I realized Doh! I totally did not need the capability (and man was I happy I realized that before trying to school pg <g>).
The danger in posting a use case is that all the ones you offer will turn out to be better handled some other way. Then you will know why your bloated fix is not in Arc.
I knew someone was going to ask for a use case, which is why I gave an example that (I thought) was so basic as not to require one: find a list of a given length. If you can't solve such a simple problem simply, that indicates to me a fundamental weakness.
Besides the kludginess of mem, consider how easy it is to screw up with find. How do you verify that a list doesn't contain any nils? Without a type system to enforce such invariants you get the worst kind of bugs: those that are manifest only once in a blue moon and present no obvious diagnosis.
That similar problems arise with other uses of nil suggests to me that we're missing an abstraction.
Let me turn this around. Is there any code that would become more verbose with my suggestion? You may think it only helps in weird edge cases (which I dispute), but what do we lose by incorporating it?
<cough> No, the /code/ you offered was looking for a list with less than a given length with the understanding that nils might be in the list. The test for a list without nils is:
(all idfn lst)
Is it a kluge to state the test positively? Actually, negative tests are understood to be harder on the cognitive system to get right.
Meanwhile, are you sure (member nil lst) is a kluge? It sure looks readable, and it sure does work. What exactly bothers you about that?
Meanwhile, a use case means your are not just running code samples you typed in at the repl, it means you are trying to solve a problem using a computer and something came up. So here we are after iterating over the members of the library and we built up a list of overdue books per member -- no, that would not have nils, that would have an assoc entry (bob . nil). Hmmm... can you help me here, I cannot think of one.
You have not responded to the very similar example of tables. Correct, one cannot distinguish the absence of a key from a key added with the value nil. A use case is not a couple of lines of code proving that, a use case is a lookup on drivers license number during a traffic stop and...?
My point you may have missed is that without a use case not perfectly well expressed with either of the two solutions I offered above, the fundamental weakness is in offering a theoretical functional gap in a language as if it were a gap in a language.
Recall that pg is already on record as wanting things that are problems in practice, that this is a hacker's language, not a strait jacket language like C++ or Java with all sorts of theoretical angst over untyped variables.
The problem, btw, of putting an air bag in the trunk of a car is the added weight. :)
Sorry (really) but I am afraid it's "just plain wrong". Variable capture is a feature, not a bug. It happens only if the macro author wants it to (on pretty rare occasions) or if they need to be fired for not using gensym to create an uninterned symbol which cannot possibly clash with another name/symbol.
As for a macro expanding into a larger lexical space where "list" has been bound to, say, 42, one of the joys of CL is never having to worry about that and we do it all the time (and its a bit of a shibboleth when we see people using "lst" in code they mention on comp.lang.lisp, we unleash the hounds who chase them over the fence to comp.lang.scheme where they belong).
I have written hundreds of macros including macro-generating macros over thirteen years and the only problem I ever has is figuring out nested backquotes, but that is because I just never managed to form a mental model of WTF is going on <g>, and it only takes me a couple of trials and errors to get past it.
There really is no problem with CL unhygienic macros. Oh, I should have mentioned that:
(let ((list 42))
(list 'hi 'mom))
... works fine and produces '(HI MOM) because CL is a Lisp-2 and shrugs off the variable name (in the operator position, variables do not apply (ugh, pun not intended). As for:
...fine, here's another fine mess you've gotten me into (when I try to compile):
; While compiling XXX in C:\DOCUME~1\Kenny\LOCALS~1\Temp\cda9402357291.cl:
Warning: Compiling a FUNCTION definition for the name LIST as a FLET. This name is in the COMMON-LISP package and defining it will be a violation for portable programs. The package COMMON-LISP has PACKAGE-DEFINITION-LOCK set, which causes the system to signal this violation.
The good news is that you are right about one thing, in CL we can use lib-x:same-name-func and lib-y:same-name-func to specify which we want should a clash arise.
And a fine argument that is. The need for hygenic macros is overstated.
What I don't see is why you can't provide both? It's not confusing to provide both, it's not difficult to provide both. There are that class of macros you write where variable capture is tedious to deal with manually, so why not save some code and make a correct implementation of hygenic macros just so that people who need them don't have to re-implement them?
I just counted up again and my code base shows 613 defmacro occurences and only 111 gensym occurences and I do not even use a with-gensyms macro, so I am not sold on the need. Then again, I do strive for the functional thang and try not even to use let. btw, I am not fastidious generally but (I guess cuz it is so easy) I never skip on a gensym -- I am definitely of the school that believes programming is hard enough as it is. :)
While we're on the subject, I know a number of people have been saying that Arc's unhygenic macros problems are a problem, because things like this break in a lisp-1:
(mac break (a b) `(list ,a ,b))
(let list nil (break 1 2)) => Error
However, you can simply unquote the function values in the macro.
(mac dont-break (a b) `(,list ,a ,b))
(let list nil (dont-break 1 2)) => (1 2)
To make this work for functions, you need to add ((procedure? s) s) to 'ac. Otherwise with arc2.tar:
arc> (mac dont-break (a b) `(,list ,a ,b))
#3(tagged mac #<procedure>)
arc> (let list nil (dont-break 1 2))
Error: "Bad object in expression #<procedure: list>"
Macros still remain a problem:
(mac list-macro parms `(,list ,@parms))
(mac break (a b) `(,list-macro ,a ,b))
(break 1 2) => Error
A problem with unquoting the function value is that if you redefine the function, any code that uses the macro will hold on to the original function as a literal. This means the solution doesn't lend itself well to exploratory programming.
Well, one could create a module system so that the macro would always see the function the way it was seen in the module where the macro was defined. Then, when the function name is rebound where the macro is applied, it doesn't affect the macro, but if the function is recompiled in its original module, it does affect the code produced by the macro. I'm not saying it's straight forward, but I do believe it's possible.
"when we see people using "lst" in code they mention on comp.lang.lisp, we unleash the hounds who chase them over the fence to comp.lang.scheme where they belong" : be careful, it's not necessarily a schemer, it might also be pg, writing a function where both l and ls are already bound...
Excellent point, it might just be an Arc welder trying to save a char -- well, we checked with the hounds and they say a CL-style defmacro and (is nil #f) -> t don't mask the Lisp-1 scent, but they loved On Lisp and Ansi Common Lisp and ViaWeb being done in CL so pg is welcome any time.
Looks like my code when it's a three-liner and it really is a bit much to be coding (in my Algebra program):
(loop for denominator in denominators...
You are right (guessing at the implicit): "list" is a terrible name for a variable unless someone really is writing a general purpose list manipulation function, but we do see "lst" quite a bit over on c.l.lisp.
btw, the real question is whether you see "list" as a variable name in the Scheme source.
Word. But I guess without built-in pattern matching or regex yer just gonna be translating Norvig's 150-line version. Fewer chars (esp. parens) but not much else.
The K version was offered when prompted, after Stevan first showed a Q version twice as long, so I guess the K version shared something mush like this from the Q version:
D:(("1 father|mother|brother|sister 2";"tell me about your 0.");
("1 am|i'm 2";"why are you 2|have you always been 2?");
("1 i 2";"why do you 2?");
("1 you 2";"we're talking about you, not me.|please don't be so personal"))
E:"|"vs"that's very interesting. do go on.|
tell me more|
i'm not sure i understand you fully|
can you elaborate on that?"
I won't spam this further, see the thread winding down now on comp.lang.lisp.
I am starting to reconsider, had an epiphany looking at the K code, realized how much friction there is in not being able to see all my code at once, and yes I have two big flat panels going. Maybe pg is on the right track. I just wish it was a Lisp-2 to elim that name collision issue.
I have to be coding pretty fast and furious in CL but it has happened a few times over the years. Nice warning or error (I forget) from the compiler, tho.
Is that because Scheme doesn't allow you to use a data item as the first item in a form? Or does it have some other mechanism currently lacking in Arc to help with this problem?
The reason (tab 'foo) doesn't work in arc is that ac (in ac.scm) effectively has a special list of macros, and when it compiles (tab 'foo), it looks up tab in the special list. What is really should do in a lisp-1 is look up tab in the same list that all the other variables are in, which is the list that (let tab (table) ...) will modify. (Yes, I know I'm simplifying. No, it doesn't matter in this case, unless you can think of a reason that it does.)
Also, there's only one list like this, and it has global scope, so you can't have local macros or anonymous macros.
Ok, so if it worked that way, then (tab 'foo) would work, but you would have shadowed the tab macro, and thus lost the ability to use it, which doesn't sound like a problem if you didn't know tab was a macro to begin with.
Anyone know if this is how Arc will work, or if the current behavior is by intent?
the problem that other people have stated is that because arc's macros are unhygienic, allowing local variables to shadow macros causes some pretty serious problems.
