The `(:provide ...)` syntax is a special extension to the Arc language for the purposes of `#lang anarki`. Every `#lang anarki` file must start with `(:provide var1 var2 var3 ...)` fo r some number of variables (even zero). It describes what exports are visible to other Racket libraries.
I believe `#lang anarki` does something to make it so `(load "plain-anarki-deep-dependency.arc")` loads the file path relative to the directory the module is in.
There are a number of awkward things about using Arc to write libraries for Racket consumption:
- Racket languages typically have a notion of phase separation where some amount of the program (the compile-time part, i.e., phases 1 and up) happens during the process of compiling a module into a .zo file, and the rest of the behavior (the run-time part, i.e., phase 0) can be performed by loading the already-compiled .zo file. Arc programs are more like `#lang racket/load` in that they only begin macroexpanding the later expressions in the file after they've run the run-time parts of the previous expressions, so it's like the whole macroexpansion process has to wait until run time. The only things `#lang anarki` does at compile time are reading the file's s-expressions and processing the `(:provide ...)` directive.
- Racket and Arc both have macro systems. They both involve writing macro definitions in source-to-source styles that usually make it easy to write a macro that abstracts over another macro call. Sometimes, in more advanced cases, these styles require more attention to get the hygiene right. However, they use different "source" types (Racket's syntax objects vs Arc's plain s-expressions), and they have rather different approaches to hygiene (Racket's lexical information carrying module imports and sets of scopes, vs Arc's gensyms and global variable redefinition warnings). This means mixing Racket macros with Arc macros is one of the advanced cases; I expect that to get the interactions right, it may be necessary to do some explicit conversions between "source" types, as well as being proficient in both languages' approaches to hygiene.
- As far as a module system goes, Arc traditionally has no way to allow multiple pieces of code to see different bindings of the same top-level variable name. (Framewarc and I believe Arc/Nu have approaches to this, but they involve writing all macros differently than before.) Hence, two Arc libraries might fail to be usable together due to mutual clobbering of the same variable, which isn't a typical situation with Racket modules. If and when modular techniques catch on for Arc, those techniques may still be challenging to reconcile with Racket's techniques, so a `#lang anarki`-based library may still not present a very familiar interface to other Racket ecosystem programmers.
- Arc is a lisp-N with its various namespaces, like `setforms`, `defcall`, and `coerce`, stored as entries in global hash tables. Racket is a lisp-N with its various namespaces, like `set!` transformers and `match` expanders, stored on the same compile-time object by implementing multiple interfaces (structure type properties/generic interfaces). This is one particular place where reconciling the modularity approaches may be difficult.
- Racket's compiler tooling tries to determine statically what files a file depends on, both so that `raco make` and `raco setup` can recompile the file if its dependencies have changed and so that if a file is bundled up with `raco distribute` or `raco exe`, its dependencies are included in the bundle. This is something I didn't tackle at all with `#lang anarki`. Things like `load` and `dynamic-require` make it difficult to keep track of dependencies statically this way.
The design I chose for `#lang anarki` was basically to get something tolerable working with as little effort as possible. There's probably a lot of room for improvement, so I didn't consider it stable enough to show off in the `anarki` Racket package documentation.
I think a somewhat better approach would involve:
- Giving Anarki a read-time namespace system like Common Lisp's, to solve Arc's inadvertent name clobbering issues (while potentially still permitting intentional clobbering).
- Going ahead and macroexpanding a whole `#lang anarki` file at compile time, evaluating nothing but the `mac` definitions at first, even though that's not the usual Arc `load` semantics. User-defined macros can sometimes be slow, making it worthwhile to expand them before producing the .zo. Only so much Arc code actually cares about running certain expressions before others are expanded, and those cases can use a CL-style `eval-when` approach.
- Assembling static information about the module just by using `eval-when` compile-time mutable definitions. Then we could dispense with `(:provide ...)` and just have a global mutable table that collects all the exports of the current module.
I think Arc's load time is pretty typical of uncompiled Racket code. In Racket, people can speed up their development process by using `raco setup --pkgs my-package` to compile their libraries or `raco make filename.rkt`
to compile individual files. That might be harder to do for Arc, especially the Arc REPL, since Arc's macroexpansion is interleaved with run-time side effects.
I'm not sure how to answer that. If you're asking what the easiest way would be to speed up initialization, it would be to load fewer libraries. Since it's an anarchic fork, we've all added lots of ideas that are good in principle but not always needed.
It's a new project. However, Hubski has been using postgres for a few years, but it's currently an amalgam of Arc and Racket. I didn't write the db code, and am trying to start fresh. The Hubski racket connection looks like:
BTW, I am using the domain xyrth.com with a nginx reverse proxy. I just had the thought it may be my nginx configuration. It probably needs to listen explicity on 5432? Here's my current sites-enabled:
Is this on hubski.com? Have y'all been using postgres there for a long time? Or are you trying it for the first time?
Can you show what command you're running to check that you "can connect to the db from the Arc command line"? I wonder if the app is able to connect but not disconnecting for some reason. That would explain the "thread took too long" timeout message.
If you've written some custom code to connect to postgres, that'd be helpful to look at as well.
Yes, it's the fork that's been maintained by the people here at Arc Forum. :) I recommend it.
Whether you want the community additions or not, you might also be interested in the "official" and "stable" branches of that repo. The "official" branch tracks the official .tar releases that you'd find on the front page here, and the "stable" branch tracks the official releases plus a few small bug fixes and quality-of-life improvements.
Penknife really was just a pile of code before now. I originally wrote it on my commute using an Android phone. Now I've finally gotten around to giving it a readme, a .gitignore, and all that. Whew. :)
There was another language I called Penknife a couple of years later because I thought of it as having the same set of design goals, so technically this one's more like Penknife Mk. I. That's what I think I'll call it.
Something about Penknife Mk. I that I often think back to is its approach to macro hygiene.
Its syntax is primarily string-based, but with the ability for other values to be embedded inside the strings. It has a kind of quasiquotaton that surrounds the quoted section with an object wrapper that the macroexpander recognizes. When the macroexpander expands that expression, it switches over to using the scope at the macro's definition site. Interpolations in the quasiquotation are surrounded with another wrapper that causes the macroexpander to switch back to the caller's scope.
When the macro is defined, its lexical scope is captured and carried on the macro's binding. That way, it's the namespace that holds other namespaces inside it. The syntax trees don't have to hold namespaces; they can just hold paths to traverse the namespace hierarchy.
I still think of this as a nice sweet spot; the code has a context-independent enough identity to be compiled, while the namespaces are mutable enough to allow REPL interaction.
Racket's sets-of-scopes approach to hygiene is mature and supports advanced features like local macros and local definition blocks, but I think of it as kind of sloppy. It spray paints one piece of information over the whole syntax tree just to mark that a variable is in scope. This approach could be handy in cases where variable scopes overlap with each other in non-hierarchical ways -- where sometimes one variable is in scope, sometimes the other, and sometimes both -- but I feel like Penknife's wrapper objects are a much tidier model of the typical hierarchical structure. There've been many moments in my Racket programming when I would have liked Racket to have Penknife Mk. I's approach to hygiene instead.
Getting back after arc is not the problem; even if I didn't have cd history set up a simple `cd -` would handle that.
Simple problem: I have a file containing a program written in Arc. I want to run it from the shell. How do I do that? I can fire up arc and `(load)` the file, but then the file needs to be in the arc2.3 dir or else I have to load it by absolute path, even if I was just in the same directory it's in.
For now I've just switched to using Anarki; I can run its `arc.sh` start script from anywhere in the filesystem and feed it a locally-relative pathname and it will run. Although it does seem to take quite a lot longer to start up than arc.
There's not much use I have for fexprs. I just had certain ideas about how they fit in with other concepts and how they could be made compatible with a compilation-favoring workflow.
I consider it basically a mistake to use the same syntax for macro calls and function calls... but not because they're different things. When we want to invoke a function like a macro, we can just conceptualize it as a macro that expands into a function call.
I think it's the same for fexprs. Macro calls can more or less expand into fexpr calls by simply preserving all the code they received under a `quote` in their expansion result. This doesn't quite get us a lexical-scope-respecting version of fexprs unless we also capture the local lexical environment, and that's not necessarily possible depending on the macro system, but it's not much of a stretch:
- Racket's macroexpander keeps track of the set of local variables in scope, but it doesn't quite expose it to user-defined macros.
- Arc's macroexpander `ac` keeps track of the set `env` of Arc local variables in scope, but it doesn't expose it to user-defined macros.
- In Guile, the local environment can be captured using (procedure-environment (lambda () '())).
- Fexpress currently lets compilation-friendly fexprs do this using the `depends-on-env?` field of a `compilation-result?` data structure. If a subexpression depends on the lexical environment this way, (fexpress-clambda ...) forms surrounding that subexpression expand differently so that they create a run-time representation of the lexical scope. This way, no one step in the code has to traverse or build up the whole environment, so the cost is spread out. Since Fexpress's variables are immutable, variable accesses don't even have to go through this run-time environment; they can be Racket variable accesses.
Anyhow, I basically consider fexprs to be one of the things a macro-capable language is theoretically capable of, even if people don't commonly prefer to use that functionality and macro systems don't always quite offer it.
I've been interested in exploring the concept of typed macros in general for the purposes of designing module systems which have both macros and typed API signatures. And I've had typed fexprs on my mind as an optimization approach since the Eight thread way back when, and I expect these two trains of thought to be on their way to the same destination.
With Fexpress, I explored the typed fexpr side, and I kept a really narrow focus on types for optimization rather than API boundary delineation so that I wouldn't make it any more complicated than it had to be. I don't want to be like "oh, by the way, in this complex type system for macros, you can find fexprs if you look for them," because some people might recoil at that. :-p And if I said something like that, but I didn't actually have an fexprs-by-default language ready to show, that would be quite a tease.
The actual languages I build with these ideas will probably not have fexpr calls as the default behavior for calling a local variable. Personally, I prefer not to have any default; it's not too much work to write out `funcall` unless I have a whole DSL's worth of local variables I'm invoking. But if it's a typed language, a lot of the local variables will probably have Lisp-1-style behavior anyway, because a variable of function type could be known to have function-call-like macro behavior and so on. And if some of the types people use turn out to declare that their variables have fexpr-like call behavior, I'll consider that to be an interesting outcome. The types, if they do any kind of soundness enforcement (unlike Fexpress's unsound hints), can keep those fexprs from sneaking into other people's code unless they're ready to receive them.
I know API enforcement wouldn't necessarily be your favorite feature in those designs. :-p But I think that's basically the picture of where Fexpress-style fexprs slot into my long-term goals, basically as one part of the possibility space that macros have more room to explore with the help of a type system.
I think macros can get pretty complicated fast. I think in some ways it's at least as hard to write well polished macros (with good error messages, editor support, debugging steppers, pretty printers, and so on) as it is to build a well polished language without macros. I think the reason it feels easy to write macros in many languages is because many languages don't set a high bar for a polished experience.
I don't think taking macros out is an answer I'm very interested in, though. I want people to be able to use my stuff without having to write code the same way I do, and vice versa. Some kind of syntactic extensibility is important to me.
And I don't think taking macros out even really eliminates the "tied to the front end" non-compositionality issues. Every abstraction that can be used has a user experience, and macros just take more control of that experience than functions do. Every time a programmer tries to report a friendly error message from a function, they're trying to take some control over that experience, but the capabilities of a function only give them so much information to work with.
A language without macros can supply a one-size-fits-all kind of polished experience, which may indeed be a much better experience than what most macros provide in practice. But a language like that also imposes a practical limit on how much more polished it can get, especially for domain-specific applications that don't have the attention of the language developers.
Thank you, that is very helpful! And this also matches some of my growing ambivalence with macros.
When I was building Wart, I repeatedly ran into situations that seemed very difficult to debug. Looking back, I tend to debug by simulating computation in my head, and macros made it easy to create code that was more complicated than I could visualize. When I built Mu I hoped that backing away from first-class macros and pervasive support for traces would help. That's still in early days, but the evidence is quite ambivalent so far.
"Were you thinking you'd have something called `map/f` that can take non-functions as its first argument?"
No, I was thinking `map` should only take a procedure.
I don't think there's a particular way to "map" an fexpr that stands out as a compelling abstraction. I propped up a certain ambitious "evaluate a list-valued expression without evaluating the list elements" vision for `map` above, but this is really more of a lazy function map than an fexpr map.
The `map` operation is basically about the list data structure. Since it's a data structure that holds its elements eagerly, we can transform it with an eager function. If it held its elements lazily, we could use a lazy function. If it held its arguments unevaluated-ly, and if the elements didn't even have to be expressions, then the abstractions we could map over these lists might be fexpr-like, but I don't know of a way to make that make sense.
If you're thinking "but then what do you do with an fexpr once you have one?" I don't know. I've only found so many compelling uses for fexprs, which have to do with places where the code of the call site is needed at run time, possibly because run time and compile time are tangled up:
- Defining custom syntaxes in mutually recursive ways that involve invoking some of the syntaxes before some of the others have finished being defined.
- Making macro systems where redefinitions of the macros automatically propagate to change the behavior of previously defined things that call them.
- Writing abstractions that take care of their own JIT behavior. In order to strategically recompile their use sites according to usage statistics collected at run time, they need access to both the source code and the ability to observe run-time conditions.
In each case, fexprs would be serving as a language front-end layer, not as internal building blocks for general-purpose computation. I think source code is second-class in an essential way; although we can write macro abstraction layers that fabricate source code as they go along, this makes it harder to report informative errors in terms of the user's original code. Macros and fexprs take source code as input, so they're tied to the language front-end in a way that procedures aren't.
And when the source code isn't something we're abstracting over much, then neither is the environment of macros or fexprs that are in scope there. (Like, the environment is more or less an annotation on the source code, and if we focus on having only one original source codebase to annotate this way, then we only have one set of annotations to produce.) So a lot of the best techniques for macros or fexprs are likely to be self-contained abstractions that keep their first-class manipulations of macros and fexprs behind the scenes, rather than whole programming styles which make pervasive use of macros or fexprs as first-class values.
Sorry, I do know that fexpr lisps can combine `map` and `and`. It sounds like you're proposing a different approach in Fexpress, so I was wondering what sort of vocabulary you are imagining. Were you thinking you'd have something called `map/f` that can take non-functions as its first argument? And so there'd be fexpr-aware variants for all higher-order functions? (Thinking harder, an `and/f` doesn't really make sense here.)