Arc Forumnew | comments | leaders | submit | rocketnia's commentslogin

This came up during ar development, although I don't remember who brought it up. I think aw was about to implement it but was concerned that it would be annoying at the REPL to have all the program's bindings be looked up eagerly (aka for them to not be "hackable"). It could be annoying for mutually recursive macros, too, although those are already tricky enough that it probably wasn't a big concern at the time.

I remember recommending an upgrade: Instead of generating:

  `(,my-func ,a ,b)
Generate something that preserves the late binding of mutable global variables, like this:

  `((',(fn () my-func)) ,a ,b)
I seem to remember aw wasn't convinced this cruft was going to be worth whatever hygiene it gained, even after I recommended a syntactic upgrade:

  `(,late.my-func ,a ,b)
Since aw values concision a lot, perhaps the issue was that most programmers would surely just write ,my-func in the hope that it would be rare (or even incorrect) to ever want to rebind it, and then other programmers would suffer from that decision.

But Pauan followed a design path like this in Nulan and/or Arc/Nu. In Pauan's languages... actually, I think I remember a couple of approaches, and I don't remember which ones were real.

One approach I think I remember is that `(my-func a b (another-func c) d) inserted mutable boxes for my-func and another-func into the result, but it would just implicitly unquote a, b, c, and d, because variables at the beginning of a list are likely to be mutable globals and variables in other positions are likely to refer to gensyms.

There might have even been an auto-gensym system in that quasiquote operator at some point.

---

I liked this approach a lot at the time. That was the same time I was working on Penknife. I was trying to avoid name collision with first-class namespaces in Penknife, but I still wanted macros to work, and I was trying to take a very similar approach. (I couldn't take exactly the same approach because my macroexpansion results were strings; instead, I think I allowed regions of the macroexpansion result to be annotated with a first-class namespace to use for name lookup.)

When Penknife's compile times were abysmally long, I started to realize that even if I found an optimization, this was going to be a problem with macros in general. Anyone can write an inefficient macro, and anyone who can put up with it can build a lot of stuff on top of it that other users won't be able to appreciate. So I started to require separate compilation in my language designs.

With separate compilation in mind as a factor for the language design, it no longer made sense to put unserializable values into the compiled code. Instead, in Penknife's case, I devised a system of namespace paths to replace them. The namespaces were still first-class values, but one thing you could do with a Penknife macro was get hold of its first-class definition-site namespace, so the macroexpanded code could refer to variables in distant namespaces by specifying a chain of names of macros to look them up from. And this kept things hackable, too, since you could mutate a macro's definition-time namespace explicitly (not that many programs or REPL sessions would bother to do that).

Not long after, I set a particular goal to make a language (Era) where the built-in functionality was indistinguishable from libraries. Builtins aren't hackable from within the language, so the module system needs to make it possible (and ideally easy) for people to write non-hackable libraries.

(Technically they only need to be non-hackable to people who don't know the source code, because once you know the source code, you know you're not dealing with builtins. I intend to take advantage of this to make the language hackable after all, but it's going to take essentially a theorem prover in the module system before it's useful to tell the module system you have the source code of a module, as opposed to just using that source code to compile a new module of your own behind the module system's back.)

Anyhow, this means I haven't put hackability at the forefront for a long time.

I think the embedding-first-class-values approach will work, and I think late binding is workable (by using late.my-func) and there's a workable variation of that late binding approach to enable separate compilation too (by using namespace paths made out of chains of macro names). So I like it, but I just have this stuff to recommend for it to make it really tick. :)

---

By the way, it's good to see you! I wondered how you were doing.

reply

3 points by waterhouse 7 days ago | link

With interpreter semantics, in which a macro gets expanded anew every time a function is called, the late binding comes for free. ;-) Then, if you want the runtime performance that comes from compilation, you optimize for the case where people are not redefining functions, and invalidate old compilation results when they do. I think that rough plan should be doable, though I haven't gotten around to implementing enough of a system to say how well it works. But I think that's the only way to get anything close to good performance in Javascript VMs (not that they expand macros, but I expect they inline function calls and such, which requires similar assumptions about global definitions), and it seems to have been done.

For separate compilation, it does seem clear that what gets serialized will be references like "the object [probably a function] named 'foo in module bar", and structures (s-expressions or otherwise) containing such references. Given that compilation implies macroexpansion, you do have to assume (or verify) that the macros from other modules are what they used to be—and that non-macros (used in functional position at least) are still non-macros. If you have a full-blown Makefile kind of build system, then by default I suppose every file's output depends on the contents of every other file that it uses; or, as an optimization, depends merely on the exact set and definitions of macros exposed from those files. (In the C++ system I encounter at work, code is separated into .cpp and .h files, and editing a .h file causes the recompilation of every .cpp file that recursively depends on it, but editing a .cpp file only causes its own recompilation. If you wanted to imitate that, I guess you'd put macros into a distinctively named set of files, and forbid exportable macros anywhere else.)

---

Thanks! I've sold out and have been working for a medium-sized company doing mostly C++ and bash (the latter is unbelievably useful) for the past 3.5 years. I make intermittent progress on the side doing other things.

reply


This is a blog post I wrote up about a bunch of programming I've been doing for my Cene language and some related Racket libraries.

I've finally been able to make an extensible `quasiquote` built on what I call hypersnippets. I've also referred to this as "higher-quasiquotation-shaped syntax" in the past, and lately I understand that my hypersnippets are the exact same thing as what people already refer to as "opetopes" in higher category theory.

Building this kind of extensible `quasiquote` operation is a goal I've been pursuing for two years now. Now I can work on polishing up the code, sorting things out, documenting them... and, ultimately, returning to the task I was doing two years ago before I had to go out of my way to figure out hypersnippets: Doing polish and documentation for the Cene language itself.

reply

3 points by akkartik 35 days ago | link

Looking forward to seeing what use you put user-defined unquote macros to!

reply

2 points by rocketnia 35 days ago | link

Thanks! :)

reply

3 points by hjek 35 days ago | link

What is a `hypertee`?

reply

2 points by rocketnia 23 hours ago | link

I'm sorry it took me so long to reply to this. Basically, I don't think I've successfully explained to anyone what a hypertee is, ever. There are some places in the comments in Punctaffy where I explain them, but I haven't put in the work to make it a very well-illustrated introduction.

Lately I realized I shouldn't be representing my higher quasiquotation/hypersnippet macro system's syntax with plain old hypertees anyway, but with something I'm calling "hypernests." So I implemented hypernests... in a flawed way that didn't actually serve the purpose I expected, so now I'm in the middle of some refactoring to fix them. It's hard for me to justify talking about the things I've built in Punctaffy when I know I can explain and motivate the topic a lot better once I have a working macro system to show for it.

It never seems like it should be that much work to just take out a piece of paper and draw up some diagrams for a blog post... but whenever I get started trying to explain like that, I usually realize I've been doing certain things wrong and need to refactor.

I hope to have something soon. I've written up a lot more unit tests, and the latest refactoring of the hypernest implementation is becoming as simple as I always hoped this kind of thing could be; the primary risk I anticipate is that it'll fall into infinite loops. I technically already have a working macro system for extensible `quasiquote` which could serve as a demo, but I'm pretty sure it breaks for operators of higher dimension than `quasiquote`, and that's what my refactoring is going to fix.

---

If it helps to write a very short explanation:

In quasiquotation syntax, the unquote operation is like a 1-dimensional closing bracket, just as the closing parenthesis is a 0-dimensional closing bracket. See, the unquoted part of the code starts at one (0-dimensional) location in the text and stops at another, so it's like a line segment. We can imagine 2-dimensional closing brackets which are shaped like quasiquotations, and so on.

The 1-dimensional closing bracket actually begins with a 0-dimensional bracket that opens a 1-dimensional region that must be closed by another 0-dimensional closing bracket.

  ,(...)
  
  The whole thing is the 1-dimensional closing bracket.
  The parenthesis at the end is the 0-dimensional bracket that closes it.
If we write a single opening bracket, including all the closing brackets it needs, and all the closing brackets those closing brackets need, etc., I'm pretty sure we have an opetopic shape as used in higher category theory: The closing brackets are the various-dimensional source cells of the opetope.

If we label each of the closing brackets (of every dimension) of the single opening bracket with a data value -- or from another point of view, put an "unquoted expression" into every hole of our higher-quasiquotation-shaped syntax, then that's what I call a hypertee.

If we have a syntax with closing brackets and (nestable) opening brackets, and we put labels on all the closing brackets of the outermost opening bracket (labels which we can think of as "unquoted expressions") and labels on all the nested opening brackets (labels which we can think of as "operators" or "macro names" which apply to those opening brackets' contents), then that's what I call a hypernest.

Closing brackets have to be of a dimension strictly lower than the bracket they're closing. That's different from opening brackets; we can nest a high-dimensional opening bracket inside of a low-dimensional one.

Nesting opening brackets are pretty exotic if you consider them from the geometric standpoint of opetopes -- how does it make sense to have a low-dimensional shape with high-dimensional faces on it? -- but it's necessary for Punctaffy's syntax purposes. That's because we need to be able to write a quasiquotation operator of some specific dimension N that can quote any operator in the language, including those of dimension N or greater. This is why I've needed to move to hypernests for syntax lately, even though I spent a lot of time thinking I could get by with hypertees.

reply


Interpreter? Are you talking about Racket's bytecode interpreter?

reply

1 point by akkartik 39 days ago | link

No, I was forgetting that ac is considered a compiler :) But it compiles the Arc codebase every single time Arc starts up. Maybe we should start memoizing its outputs to disk somehow, see if that makes it noticeably faster. My suspicion is egregious runtime processing like ar-denil-last and ar-apply-args will cause it to not make a difference.

Given the pervasiveness with which Arc has to make such transforms at runtime, I've gotten into the habit of thinking of it effectively as an interpreter.

reply

4 points by rocketnia 40 days ago | link | parent | on: Installation on windows?

I've been pretty shy about it, but I finally sent a tweet three years ago.

https://twitter.com/rocketnia/status/658565199402692608

reply

3 points by akkartik 39 days ago | link

Tweets can be missed. Perhaps we should email hn@ycombinator.com.

I've been reluctant to do this, because the outcome may well be, "wait, is this old site really still up? Let's just take it down." :) Don't mind li'l ol' us out here, we're no trouble, no trouble at all..

reply

2 points by hjek 39 days ago | link

> "wait, is this old site really still up? Let's just take it down."

That is a valid concern. Perhaps, we could ask that if YC chooses to shut it down, could they at least give someone here a copy of the `www` folder?

Then someone could take over hosting this forum (which possibly might be on a different domain).

I have a somewhat reliable server running anyway, and it wouldn't be a problem for me to do this, but if someone else could do it, that would also be great.

But yes, worst case scenario is that this site is nuked. But that would just be such a disrespectful response, so I kinda doubt it..?

reply

3 points by i4cu 39 days ago | link

If someone does email, please request that the main page[1] be updated with a link to anarki as opposed to just a change in install instructions.

1. http://www.arclanguage.org/

edit: I'm ok if they take it down. I'll know where to go via the anarki wiki (if someone updates it). And it may actually be better if they do take it down IMHO as it will force everyone to find a place with more control over the setup.

reply

3 points by krapp 39 days ago | link

I mean... would it be so bad for the Anarki community to be hosted on a forum running Anarki?

reply

2 points by hjek 39 days ago | link

Then we would be even able to actually search stuff on the site!

reply

3 points by shader 39 days ago | link

I think if we fork the community site to run on anarki. which I think is more likely than being given control over the Arc Forum, we should consider ways to archive and bring forward all of the stuff on the existing arc forum. It shouldn't be too hard to crawl the forum, though I think there might be some DoS prevention that would slow it down.

reply

3 points by rocketnia 40 days ago | link | parent | on: Anarki Reference Documentation

Yeah, these would be great! I focused on shipping something, but there are many ways it can be improved. :)

reply

5 points by rocketnia 44 days ago | link | parent | on: Anarki Reference Documentation

Over the past week, I set up some HTML generation code and a deployment script on Anarki that creates this documentation page based on Anarki's (help ...) information.

I used this GitHub issue as a journal of my progress: https://github.com/arclanguage/anarki/issues/112

There are several more things I should explain about this:

-

===== What information a help entry is based on =====

Sometimes, some help information is populated while other information isn't available. For instance, right now `list` has everything help.arc is designed to display except for a docstring:

  arc> (help list)
  [fn]  (list . args)
  list is not documented.
  
  Examples:
    arc> (list 1 2 3)
    (1 2 3)
    arc> (list "a" '(1 2) 3)
    (a (1 2) 3)
  nil
  arc> (src list)
  (from "arc.arc")
  (def list args args)
  nil
It has a signature ('args), an implementation definer ('def), an implementation body ('args), a source file ("arc.arc"), and examples ('((list 1 2 3) (1 2 3) (list "a" '(1 2) 3) ("a" (1 2) 3))), and of course a value that has a type ('fn), but it has no docstring.

When the code in build-web-help.arc generates the HTML page, it determines whether to display a help entry purely by whether it has a docstring or not. I figure this is a good way to distinguish between things that are interesting to read about and things that are idiosyncratic helper functions, but one consequence is that there is no entry displayed for `list` right now.

-

===== Broken links =====

When the documentation refers to another entry by [[foo]], it's converted to an link to the relevant entry on the page. If there is no entry by that name on the page, it's instead converted to a span of style "broken-link", which shows up in red. If you'd like to fill in gaps in the documentation, you can view the source of the page to find all the occurrences of "broken-link".

-

===== Security of repository access privileges =====

The script pushes to Anarki's gh-pages repo using a personal access token for my GitHub machine user, rocketniabot. I just created rocketniabot for this purpose. The token is limited to pushing to rocketniabot's public repos, and right now the only public repo rocketniabot has access to is Anarki. If this changes in the future, I might want to ask someone else if they can use an access token. (And if no one wants to volunteer one, it's not the end of the world; we'll just stop having automatic pushes to the `gh-pages` branch until it's fixed again.)

The token is not committed to the repo; it's set up in the Travis CI settings as a so-called "encrypted environment variable," which is only exposed during a non-pull-request build or a pull request build that comes from another branch in the same repo. I believe this prevents non-contributors from accessing the token.

Although people could make rocketniabot look bad by having it push abusive content to the `gh-pages` branch, they would first have to either push a build script that exposes the token or push content like that to the `master` branch themselves.

If we notice either of those kinds of abuse occurring, I recommend we take two actions:

- Remove that contributor from the Anarki project on GitHub so they can't keep doing this.

- Please let me know so I can revoke the compromised rocketniabot access token.

- Until I do that, remove the rocketniabot contributor from the Anarki project.

- If you'd like to set up automatic `gh-pages` pushes again and I'm not responding to messages, then I recommend you choose another user account who's willing to be responsible for the automatic pushes, have that account set up a personal access token with public repo access, and put that token in the Travis CI configuration instead. (If you do this, please change the variable name so it's not "ROCKETNIABOT_GH_TOKEN". I think we need to keep track of whose it is so we can notify the right person when it's compromised.)

-

===== Security of the website's client-side data =====

If we ever have any page on the arclanguage.github.io domain store client-side data (like localStorage entries or cookies), someone could potentially access this information and take advantage of it before they're removed from the project. Because of this, I recommend we don't store any client-side data on that domain.

reply

3 points by rocketnia 48 days ago | link | parent | on: Separation of concerns

Please be patient with krapp, hjek.

You also made a bunch of breaking changes the repo a while ago. Because of those, I got more of the tests running under the Travis CI script to help us all monitor them without having to ask each other to remember to run them all the time. I really hope I didn't make you feel ashamed about that. You got to make changes, and I got to make changes, and I felt like the repo was better with both of those together. The same is true here.

reply

3 points by hjek 48 days ago | link

I did break some things in Anarki, and I appreciate you spending time helping me fix those breakages and adding more automated tests. Thank you.

What bugs me here is not so much that stuff gets broken now and then, but this nonchalant attitude to breaking things, and lack of acknowledgement for others that have to wipe up the mess.

> And... people just have to fix it once.

> Given that is not a mature language or a stable framework yet, I would disagree that breaking changes, per se, should be avoided as a matter of course.

Also, I have a strong aversion to CAPTCHAs, partly because they're non-free Google SaSS, but also just because they are just plain annoying. Why would anyone want to inflict that upon themselves? There's already plenty of ways to deal with spammers in News.

And, quite unrelated, I'm not big on Github as their site requires non-free JS, and I'm particularly wary of them since they were bought by MS, so that is just one more reason to host the code elsewhere. So, I've forked Anarki, and we'll see what happens: https://notabug.org/hjek/knark

reply

3 points by akkartik 47 days ago | link

I agree that we should hold ourselves to some standard about not breaking Anarki for each other.

It's our job, yours and mine and that of other long-time participants, to explain the norms that we have here. And to do it in a nice way, realizing that what's obvious to us isn't necessarily obvious to newcomers.

This is a pretty non-standard repo where anybody can make changes, so it's understandable that a newcomer may think stability is not important. We have to articulate precisely what sorts of stability are necessary and what we don't consider important. It's a tough job. So I really hope you stick around to help us out :)

I'm actually curious how you run a production service on Anarki. Everytime I ever tried to do that I would maintain a cutout in between that was wholly in my control, and decide what patches should flow from Anarki to it.

reply

2 points by hjek 47 days ago | link

> So I really hope you stick around to help us out :)

Definitely!

> I'm actually curious how you run a production service on Anarki. Everytime I ever tried to do that I would maintain a cutout in between that was wholly in my control, and decide what patches should flow from Anarki to it.

Usually I'm trying out the latest commits on a local laptop before doing git pull on the server. Works fine. If it's broken for some reason, then I'll just do a `git reset --hard [known-good-revision]`, because the `www` folder with all the news data is outside the git tree anyway so it stays as it is (and no-one have yet been changing the data format, so that's not an issue either.)

I find it a lot simpler to manage than for example Wordpress (although Wordpress on the other hand has tons more featuers and plugins).

reply

3 points by krapp 47 days ago | link

>What bugs me here is not so much that stuff gets broken now and then, but this nonchalant attitude to breaking things, and lack of acknowledgement for others that have to wipe up the mess.

You don't have to wipe up the mess. I didn't ask you to wipe up the mess, and I didn't expect you to. I also suspect you're conflating my beliefs about whether or not Anarki should be willing to make breaking changes when necessary with some general lack of regard for whether or not anything works at all, and that's not the case.

I'm new to lisps, to this lisp in particular, to this framework, and to working in a team. I admit (and have admitted) that I make mistakes and I've picked up some bad habits that I need to correct. If I know about an issue, I will gladly deal with it.

But, as to the specific issues you listed... I don't know what happened with the css (it worked for me when I pushed it but your comment that it might have needed testing in other browsers is correct,) but #104 wasn't broken, but misconfigured. And you've explained at length already why that entire commit was misguided, but it wasn't broken.

But sometimes I do miss things, forget things, there are some mistakes I might not even see because I'm working on Windows. But I am trying to make things better here. So thank you for cleaning up after me, and I will try harder in the future.

That said, I stand by moving news, and I stand by my statement that we should be making breaking changes (with the caveat that those changes improve the code.) Now is the time to do that, when there are so few people actually using this codebase, as opposed to later when there might be dozens or hundreds, or never, simply because change is difficult to deal with. Chances are most of the entire universe of people using this fork in production are already in this thread. Moving news to a new folder shouldn't be a reason to create a hostile fork.

And that's why I agreed with i4cu's suggestion earler that a stable branch might be a good idea... which for some reason you interpreted as a personal attack,

    "(...) and then suggesting that I should just 
    get off the master branch if I don't like your "experimentation"."
Rather than more charitably, that you should stay on the master branch and the unstable branch should be elsewhere, as I was actually conceding your point and agreeing with what I thought would be an equitable solution.

reply

2 points by hjek 45 days ago | link

> You don't have to wipe up the mess. I didn't ask you to wipe up the mess, and I didn't expect you to.

I agree. That was my own expectation, and also I might have been wrong in expecting stability from a repo that's meant to be "extremely permissive in accepting patches."

> And you've explained at length already why that entire commit was misguided, but it wasn't broken.

That I don't agree with. I would consider a commit, that makes all links not work in the default config and prepends the site name to already absolute links, broken as well as misguided.

> But I am trying to make things better here. So thank you for cleaning up after me, and I will try harder in the future.

That's great. It's good to see work done on Arc. Looking forward to see where this goes.

> Moving news to a new folder shouldn't be a reason to create a hostile fork.

Let's just call it a fork, not a hostile fork, shall we? Also there's some other reasons listed in the readme of the fork.

reply

3 points by i4cu 48 days ago | link

> Also, I have a strong aversion to CAPTCHAs, partly because they're non-free Google SaSS, but also just because they are just plain annoying. Why would anyone want to inflict that upon themselves? There's already plenty of ways to deal with spammers in News.

Languages allow people to make their own choices on what is worthy to build. Debates like this are why it's a shitty idea to have apps baked into the language.

reply

3 points by hjek 47 days ago | link

True. The tight coupling between Arc and News can in some cases be a problem.

It goes a bit further than the folder layout though, and I find the border between app.arc and news.arc a bit murky.

reply

4 points by i4cu 47 days ago | link

> I find the border between app.arc and news.arc a bit murky.

It could just be the name that's offsetting you, but app.arc is an application library just like srv.arc is, only manages application ops/state instead of server ops/state. The best way to look at it is that app.arc, like srv.arc, can optionally be used by different applications (like news.arc, blog.arc or whatever), which is why it's under lib. Someone who wants to build an app, needs to know what libs can be used and those can be. They also need to know that news.arc is not a library to to use for a new shiny app they want to build and when they want to share their app where to put it. Right now the only place is under app/, and ideally (IMHO) we can move them out further someday.

reply

4 points by hjek 46 days ago | link

Agreed. Yet it is also possible to imagine apps that could benefit from tighter coupling with News, e.g. to access user 'karma' and to integrate into the layout and navigation of News.

reply

3 points by i4cu 46 days ago | link

> to access user 'karma'

That, to me, is a good argument for a karma.arc library.

> to integrate into the layout and navigation of News.

which is an app. :)

reply

3 points by hjek 45 days ago | link

Interesting idea.

Also, the password reset email function in app.arc requires `this-site` to be set, but `this-site` is set in news.arc. I'm not sure how I'd go about fixing that.

reply

4 points by i4cu 45 days ago | link

I hadn't realized that arc could even point to an unbound variable and still have the file load without errors. Oh my.

In Clojure there are namespaces and I can rebind vars like such:

  (alter-var-root #'app.arc/site-url* (constantly "http://news.example.com/"))
So I could just put values (like site-url*) into app.arc with default settings and reset them from news.arc. Provided that it's utilized dynamically then all is good.

However lacking this functionality in arc I would just move the required config settings to app.arc and be done with it. You're still loading app.arc at the top of news.arc so the only real problem is app users need to know to change the settings therein (a comment would do).

reply

2 points by krapp 46 days ago | link

>which is an app. :)

News already has a repl and "apps" in prompt.arc, and I'm already working on getting them to work together. That could provide the basis for a plugin (app, what have you) system.

reply

5 points by rocketnia 49 days ago | link | parent | on: Separation of concerns

Fixing other people's test breakage is one of the only things I can do for Anarki that doesn't make me feel like I'm breaking things myself. :-p

reply

4 points by zck 47 days ago | link

Well, it's a great help! Writing tests can help too, but just being around and helping is wonderful.

reply


Is sounds like most of what you're talking about is quasiquotation. (Maybe a lazy quasiquotation?)

For this:

  (cons 1 (cons (car x) (cdr y)))
You're writing this:

  (1 (/ car x) / cdr y)
The same thing with quasiquotation would usually be:

  `(1 ,(car x) . ,(cdr y))
And that's sugar for this (possibly using different words on different Lisp dialects):

  (quasiquote (1 (unquote (car x)) . (unquote (cdr y))))
And then `quasiquote` itself is a macro. It's an interesting macro to implement, and I just spent like two years trying to streamline the way I was implementing it for my languages 'cause I kept adding bells and whistles to it. :)

If you write a slightly different macro where the syntax for (unquote ___) is (/ . ___), then you could use it like this, which matches the syntax of your example exactly:

  (t-expr (1 (/ car x) / cdr y))
I think the advantages of quasiquotation specifically shine when the quoted data is also program code in the same language (or like you just said, a DSL). In that specific case, I like that the code I'm writing looks the same regardless of whether it's under an unquote or not. There's a practical advantage in terms of being able to copy the code back and forth as necessary.

It sounds like you're looking at this as a data encoding? A lot of data encodings take pains not to be Turing-complete. Then again, for inexpressive languages like HTML, there are a lot of Turing-complete templating languages designed to dynamically generate them.

HTML has other languages embedded in it like JavaScript, and it sounds like that's relevant to what you're going for here. You're going for a language which can begin running even though not all the syntax is computed yet. That's pretty interesting. (Actually, maybe I'm projecting, 'cause I've been dreaming of something like that too: A language which multiple people collaborate on, which can begin running before all of the code has been written.)

You can of course experiment with that in most Lisp dialects using a quasiquoted term where all the unquotes have functions in them:

  (my-interpreter `(1 ,(lambda () (car x)) . ,(lambda (cdr y))))
And you could write a macro that puts in those lambdas automatically so it's not so verbose. Then your parser could use the Lisp's reader, wrap the result in (t-expr ...), and eval it in an appropriate namespace to yield your compiled program-syntax-with-functions-in-it. Then you could pass that syntax through an interpreter.

Is this all making sense? XD I started going through ideas a little fast, and maybe I missed your point a long time ago.

reply

3 points by rocketnia 51 days ago | link | parent | on: Separation of concerns

Have you seen lib/ns.arc.t? If you want more examples than that... could you give examples of what examples you'd like to see?

Keep in mind that if you use ns.arc to load an Arc library in a namespace, you'll usually have to work around problems with unhygienic macros on a case-by-case basis. I don't think this would typically be any easier than working around whatever problems were preventing you from loading it in the main namespace.

When I made ns.arc, I did make a few changes to Anarki so it would cooperate a little better with Racket namespace manipulation. When I made Anarki fit into the Racket ecosystem with `raco install anarki` and `(require anarki)` and `#lang anarki` support, I made several more changes that made it possible to do things like load more than one instance of the Anarki built-ins in separate Racket namespaces. However, ns.arc is still primarily a library for manipulating Racket namespace objects. Any use it has as a library isolation tool is still aspirational.

reply

2 points by krapp 51 days ago | link

One persistent problem is that the HTML macros have to work around everything else, and as a result, the implementation is haphazard.

You can't build an HTML table with (table) for instance, it has to be (tab)... but just ignore that (tab) doesn't print \t. I accidentally broke things adding a (body) macro because other macros were using that for expansion. The best possible world would be one in which every HTML macro matched its corresponding HTML tag, or if Anarki supported XML syntax natively. Without some kind of namespacing or scoping the former would be impossible, and I suspect the latter would annoy Lisp purists.

reply

2 points by rocketnia 50 days ago | link

I'm really glad you have an opinion about what the HTML library should do. Without bold, breaking commits like yours and hjek's, Anarki doesn't go anywhere. They're pretty much the only way Anarki gets maintained at all.

What you're saying would make sense if html.arc were a self-contained library for public use. Why does it have shortcuts for some HTML tags but not others? Someone in the public might want to use those others!

I think I'd approach this differently: Every client of html.arc can already say (tag foo ...) or (tag (foo ...) ...) to write an HTML tag. Verbosity is sometimes valuable; it can let us grep the codebase for something like "tag" to find all the places HTML tags are generated. So if the shortcuts of html.arc are causing distress over their inconsistency, I would remove all of them... from html.arc, at least.

Sometimes brevity is valuable too. It helps us fit a bunch of code on one page, for instance. So those things I would remove from html.arc could still belong in another file. Or maybe a few other files; sometimes utilities are self-consistent on their own but are inconsistent when viewed as part of a group.

I think your vision of how these shortcuts should work is self-consistent, and it would even work well as a Racket library. In Racket, we can use `(require (only-in ...))` or `(require (except-in ...))` to require some identifiers from a library without getting identifiers that clobber the ones we're using for other purposes. And we can use `(require (prefix-in foo: ...))` to require a library's entire set of exports under a prefix.

Anarki doesn't have Racket's `only-in`, `except-in`, or `prefix-in` abilities; it only has the ability to load everything in a file. But Anarki users can achieve something similar: The library itself can use more verbose names like `body>` or something, and users can write `(= body body>)` if they want to get rid of the verbosity.

---

I think I've actually seen that `body>` naming convention somewhere, but I can't find it. While I was looking for it, I noticed a similar level of brevity in almkglor's whtml.arc on the old Arc 2 Anarki branch:

https://github.com/arclanguage/anarki/blob/arc2.master/whtml...

  (w/html
    ('head ('title (prn "Page")))
    ('(body style "font-size: 200%")
      (prn "HELLO!")))
Scoped DSL techniques like this one can be a nice way to get some greppability (searching for "w/html") while also getting some brevity (shaving three characters off of "tag " to arrive at "'").

reply

1 point by krapp 49 days ago | link

>What you're saying would make sense if html.arc were a self-contained library for public use.

The thing is, it is, whether it's meant to be or not. Arc's documentation describes it under the heading "html generation" (in general.) It's in what appears to be Arc's "standard library" /lib, so any newcomer is going to see it there and think that's what they should use when they want to generate html. And it has the rather canonical-seeming name of "html.arc."

If html.arc isn't supposed to be Arc's general purpose library for HTML generation then what do we call it when we make it?

I think discussions about APIs and interfaces are missing the confusion (which, admittedly, may just be mine) over what exactly /lib is supposed to be for?

>I noticed a similar level of brevity in almkglor's whtml.arc on the old Arc 2 Anarki branch:

I really like it... but wow am I not capable of even reading that code ._.

reply

3 points by rocketnia 49 days ago | link

"Arc's documentation describes it under the heading "html generation" (in general.)"

That documentation is for Anarki Stable, the branch meant for users who aren't thrilled with Anarki's changes and just want to use official Arc, but who would rather not have to fix official Arc's known bugs on their own. On that branch, html.arc probably is the only HTML-generation library we can or should document.

Despite not being written as documentation for the Anarki master branch, it's nevertheless some of the most accessible documentation for that purpose. I see that's pretty frustrating. It's good that you're mentioning it or no one would work on it.

Yesterday, since this was on my mind thanks to these threads, I wrote some code for generating a simple HTML page based on Anarki's (help foo) documentation. Once I get this to automatically deploy with each new Anarki commit, it should give us a much more up-to-date documentation resource to link to.

---

"any newcomer is going to see it there and think that's what they should use when they want to generate html"

It pretty much is! Even if it's not as good as you can imagine it being, it's still the go-to HTML generation library for Anarki.

When I implied it wasn't "a self-contained library for public use," what I meant was that it was primarily made to get the webapps to work. It's quite a non-minimalistic approach to HTML generation if you ask me -- I'd rather treat every attribute uniformly rather than having a big table of special cases -- but in other places it does only what it needs to do for the purposes of those applications.

Paul Graham pursued the web server code in order to put pressure on the "core language" in arc.arc, to make sure he was focusing on operators that were actually good for something. He released the language to the public so he could put a wider variety of pressure on arc.arc. The operators in arc.arc were never finished, so html.arc was probably a long way from being finished itself.

---

"If html.arc isn't supposed to be Arc's general purpose library for HTML generation then what do we call it when we make it?"

If someone makes a new library for HTML generation, we can potentially embrace that as the new html.arc. Anarki is unstable enough that this would be pretty normal (although I think people have had a polite tendency not to delete something in case someone else was still attached to it).

---

"[paraphrased] What is /lib for?"

Personally, I'm reluctant to call it a "standard library" without a standards document, but it does "come standard" with Anarki, so...

Of the 14 files in the release of arc0.tar, the libs.arc file contained this:

  (map load '("strings.arc"
              "pprint.arc"
              "code.arc"
              "html.arc"
              "srv.arc"
              "app.arc"
              "prompt.arc"))
So essentially half of the files in the first Arc release were essentially lib/ files that just hadn't necessitated their own folder yet.

Looks like there was always a lib/ directory in Anarki, even as far back as the first Git commit. At this point it's the obvious place to put files that some people might be interested to (load ...) and some people wouldn't care about. If someone has a single .arc file to share, dumping it there is easy regardless of what state it's in, and it usually fits right in.

---

"I really like it... but wow am I not capable of even reading that code ._."

Aw, I bet you could! :) Do you want to? Maybe if you've got a specific piece of code that perplexes you, we can help out.

Anyhow, one of the only reasons so many things are still stuck in the "arc2.master" branch is that porting them would be too much work. It could be that whtml.arc's time for porting has arrived. :-p

reply

2 points by krapp 49 days ago | link

>Aw, I bet you could! :) Do you want to? Maybe if you've got a specific piece of code that perplexes you, we can help out.

I'm used to more verbose languages and the redundancy of heavier syntax. Sometimes Arc reads to me like a diagram sketched on a napkin.

>It could be that whtml.arc's time for porting has arrived. :-p

To me it still feels like a kludge for the actual problem, which is a lack of namespaces. Maybe it's time to try out ns.arc?

reply

3 points by rocketnia 49 days ago | link

Right now, the only things ns.arc can import into local scopes are non-macro values. (Or rather, it can import macro values, but it determines what value they have well after the code it's in has been macroexpanded, so you'll just see them as (annotate 'mac ...) values, and calling them will be a function call that fails with an error.)

What you want are local macros (Common Lisp's `macrolet`, Racket's `let-syntax`). Suppose we name Arc's equivalent `w/mac`.

Then (w/html ...) could expand to this:

  (w/mac html tag-html
         head tag-head
         body tag-body
         ...
    ...)
To implement `w/mac` requires changes to the macroexpander in ac.rkt. I don't know how much this will make sense to you, but this is the approach for anyone who wants to take it: It requires a change to the way `ac-call` looks up the macro's value (`ac-macro?`) so that it can look it up from the local scope (the `env` parameter). This in turn means the local scope value `env` needs to be refactored so it's a hash table rather than a list, and then `w/mac` can look it up from that table. The `env` parameter isn't passed into macros, so either `w/mac` would need to be implemented as a special form or we'd need to come up with a way that macros can optionally declare that they want to receive an `env` parameter.

This would be something of a breaking change to Anarki because it means Anarki will now allow a local variable to shadow a global macro. That was the problem you encountered with (body) in the first place; some calls to local variables were expanding as macro calls now because the macro lookup ignored local variables altogether.

There might be some code out there of the form

  (let foo 1
    (bar foo))
where (bar ...) is a macro call that expands to (foo ...) for some global macro `foo`. This code would now fail.

For some time I've thought it would be worth it for the local macro support, but I'm too afraid to make the breaking changes.

Nevertheless, a while back, there was a consensus here that the way things worked already was a bug. See "Global macro names take precedence over local lexical variables" at [1]. I don't remember if this bug was ever fixed in Anarki; it doesn't seem to be fixed now. (Maybe there was another consensus to reverse it? It could happen.) It's the very same bug you encountered with (body) in the first place.

[1] https://sites.google.com/site/arclanguagewiki/arc-3_1/known-...

reply

2 points by akkartik 48 days ago | link

lib/tem.arc was extracted out of the original codebase. There may be other examples like that.

reply

3 points by rocketnia 48 days ago | link

Hmm, it doesn't look like tem.arc was in any of Anarki's versions of the official releases. Didn't you factor out that file?

reply

2 points by akkartik 47 days ago | link

Yeah, that's what I meant :) but I was lax in my phrasing.

https://github.com/arclanguage/anarki/commit/2820bbf7ee

reply

2 points by rocketnia 47 days ago | link

I'm not sure what you're saying it's an example of...

reply

2 points by i4cu 47 days ago | link

Your comment detailed what the libs.arc file contained.

the code within tem.arc was originally part-in-parcel within those files, but was extracted out into its' own file. So he was just commenting that there is other code that exists in anarki that was originally included and that there may be more examples of such.

reply

3 points by akkartik 47 days ago | link

Right. It's not the case that everything in libs/ that isn't an original filename is new code.

It was just a minor nit :)

reply

3 points by rocketnia 46 days ago | link

Ah, I didn't even know I was implying that. Thanks for clarifying. :)

reply

4 points by i4cu 46 days ago | link

Anytime you need us to let you know what you're thinking just let us know. We got you covered.

reply

3 points by akkartik 46 days ago | link

😂

I was responding to this:

"Of the 14 files in the release of arc0.tar, the libs.arc file contained this:

  (map load '("strings.arc"
              "pprint.arc"
              "code.arc"
              "html.arc"
              "srv.arc"
              "app.arc"
              "prompt.arc"))
So essentially half of the files in the first Arc release were essentially lib/ files that just hadn't necessitated their own folder yet."

But rereading it now I think I was responding to the contrapositive of what rocketnia said :)

reply

3 points by krapp 46 days ago | link

Is that an emoji?!

reply

More