I stopped at spliceable-list (in scan-past) which needed lib/spliceable-list.arc. I didn't want to go down the rabbit hole of loading it after I saw a redefinition of append.
The dependencies might not be incompatible but I didn't try.
But yes, it might make sense to avoid pulling in spliceable-list. As long as you can see what it's being used for, there are other ways to get the behavior it provides, though the easy alternatives will be slower.
But as you're finding out, it's work. You can't just pull in dependencies willy-nilly. Fundamentally, arc3.1 and anarki are both about sharing code as ideas, not code as functionality. We've repeatedly mentioned how neither makes promises of compatibility. You can't treat them as interchangeable lego blocks and expect them to just snap into place. No matter how similar they look, they're not close to each other. They're really more like separate universes that you're 'wormholing' matter and energy through. Start with the assumption that that will cause some annihilation unless you're willing to be careful and detail-oriented. How much time should it take? Maybe "dawn to midnight" is reasonable. There's no way to tell until you try to do it. You're in terra incognita with every single little task like this.
Anarki tries to make such wormholing easier by providing tests. Tests are a better "unit of transplanting" than functions or something else. Try to first understand what's involved in making a test pass, then copy the test over, then make it pass. Think of the file upload code in anarki as merely an exposition to help guide you when creating your own version.
But this is harder when transplanting to arc3.1 because it has no tests. Honestly I have no idea why you stick with it. Imagine I'm a villain in a superhero movie or a bollywood movie: "where is your savior now? How long will you be faithful to him?" :) Stay with one repo and you'll be out on a limb less. Pick a repo with tests, and you won't be locked in to it when you do need to go out on a limb and transplant code from somebody else.
I feel bad that you struggled for so long by yourself. If you'd ask for help sooner, and if you provide more information, I think your path could be much eased. For starters, it sounds like you've added to arc3.1 over the years. It makes it harder for me to be sure that any code I suggest won't harm you. I recently made a suggestion to tvvocold over email that caused problems because it turned out he'd made changes I wasn't aware of. I'd prefer not to have more of that on my conscience. Can you share the current state of your codebase?
The final thing that helps us help you: forget about compatibility. Programming gets harder the more constraints you add, so try to focus on concrete issues rather than imaginary abstractions. How do you know your existing changes to arc 3.1 are "compatible"? If someone not called pg magically included file upload atop arc3.1, how would you be sure it was "compatible"? In my opinion the reason that "sending anything to a server requires this whole complicated way of multipart encoding/decoding stuff" is the expectation that things just work when you transplant them from one setting to another. I'll make a moral judgement as kindly and respectfully as I can: To the extent that you care about compatibility, you're part of the problem. Uploads are complicated because somebody else decided they didn't want to know about the details and just wanted uploads to work. Add thousands of such people with slightly different scenarios and vague fears of breaking each other, and you end up with all this complexity.
This is your main point really. Compatibility is nowhere close in importance.
I stick with Arc because one of its premises is following an axiomatic approach to building up a language. I don't know enough about Anarki to know if it does the same thing. I could be poorly informed. An admittedly casual survey of the source didn't make me confident that it does. I thought Anarki had logic it didn't need. It was also suspicious how little I ended up needing to add to Arc 3.1, at least until the issue of uploads.
I also stick with Arc because HN is written in it while I don't know of applications written in Anarki. I don't know if even Anarki's primary contributor uses it for web apps (do you use Anarki?) If they don't, that's a dangerous sign of building out of love for building, not need. It can be a good way to explore ideas, but not as good of a way to pick a language to write a web application in.
I do see the savior argument though. Pg admits Arc is missing a lot and doesn't recommend people use it, and he's the one who wrote it.
> Pick a repo with tests, and you won't be locked in to it
What also prevents someone from being locked in is compatibility, not just tests. Having tests for an incompatible language doesn't help a user when switching the language. They have to rewrite their program, and what they have in their head is the source of their program, not the source of the language. It's harder to switch.
> How do you know your existing changes to arc 3.1 are "compatible"?
They added to Arc 3.1, but they did not remove from it or modify semantics.
> If someone not called pg magically included file upload atop arc3.1, how would you be sure it was "compatible"?
I wouldn't be sure, it would need testing. But I know without testing Anarki is not compatible with Arc 3.1.
If pg released Arc 3.2, I know I only need to modify my application to account for the diff between Arc 3.1 to Arc 3.2. That should be a smaller changeset than the changeset between Arc3.1 to Anarki.
> The final thing that helps us help you: forget about compatibility.
This isn't your main argument. Whether I used Arc 3.1 or Anarki, the compatibility issue would still be an issue. I agree that breaking compatibility helps, by the way. Me complaining about multipart data in HTTP is, you guessed it, a result of not breaking compatibility. If there was a new version of HTTP that was not compatible with the older ones but communicated with sexps, I'd switch.
But what you really want from me is to use Anarki.
What I really want from you is exactly nothing. I couldn't give a rat's ass what you choose. I'm pointing out that the chains you find yourself shackled in are of your own creation, in your own imagination.
I mean, I did say I'd help you with whatever your repo was, didn't I? What an ungracious insinuation!
> If pg released Arc 3.2, I know I only need to modify my application to account for the diff between Arc 3.1 to Arc 3.2. That should be a smaller changeset than the changeset between Arc3.1 to Anarki.
You're making some big assumptions about Paul Graham's lack of productivity in the last five years.
> They added to Arc 3.1, but they did not remove from it or modify semantics.
If you define a new function and arc 3.2 happens to define the same name, is that incompatible?
> I also stick with Arc because HN is written in it while I don't know of applications written in Anarki.
I don't understand. Anarki provides the HN codebase just like the arc codebase that constitutes 99% of its code.
> What also prevents someone from being locked in is compatibility, not just tests. Having tests for an incompatible language doesn't help a user when switching the language.
I have no idea what the words you use mean. Tests help you if you're willing to, you know, open a file and write code to make them pass.
This is an unproductive discussion. I'll leave you to keep searching for "compatible" additions to a language that never promised compatibility, where that very concept is utterly meaningless.
I'm sorry I insinuated you want something from me. You've done a lot to selflessly help Arc since it was released, probably more than its own authors and I appreciate that. I'm also sorry I upset you.
I'm unable to reproduce it now. I disabled the bug fix, and I believe I am now running the same code as I did when the bug occurred, but I no longer get this exception when I visit that url (it's an ordinary GET request without parameters). The state of the app has also changed and I wonder if this state change has something to do with it.
What's certain is that it's a call that passes 3 arguments to cut. I had logged the exact call to cut to be:
Yeah, none of those call sites seem responsible, as far as I can see. The calls in markdown and unmarkdown only trigger if the string starts with certain chars, which it doesn't. Remind me, are you using anarki?
Ah, you're using arc 3.1 without the mutable-pair bugfix: http://arclanguage.org/item?id=18070. Anything can happen after a race condition. Hold on, let me patch that bug to the stable branch so you can pick it up.
Edit 1 hour later: Done, https://github.com/arclanguage/anarki/commit/b683a84a68. lark, can you start using this stable branch? Should be a lot less disruptive than migrating to anarki, and will also help us better help you.
Thank you for tracking down and pointing out the mutable pairs issue.
I wish there was a version of Arc that made it clear what was changed in it, and made it easy to not apply some changes. A version that consisted only of:
- the original, untouched, Arc 3.1 -- however broken it might be.
- a single Arc file called fixes.arc that contains all fixes needed as redefinitions (say, the bugfix for cut). This is different than modifying the originals; you always know what was modified.
- a collection of .patch files, to patch what can't be provided in fixes.arc (like ac.scm)
- a script called arc.sh that (a) deletes all patched files, (b) creates new copies of the original files (untouched), (c) patches them, (d) starts Arc.
About the github commit link, I don't understand what's going on with it:
- I don't know what kind of branch this is -- is this Arc 3.1 or Anarki? I want to keep using Arc 3.1.
- If it's Arc 3.1 why is it named Anarki?
- Is there a way to get a diff? If not, how do I extract this branch and then get a diff (do I need a different link, and if so which one?) Is there a way to not use git?
(Couldn't this be easier? Why is there so much work involved? I think I want a new way to organize programs.)
You know, you don't have to switch to the entire 'stable' branch. Just apply the patch linked to in my last comment. It doesn't rely on anything outside arc3.1.
Anarki has multiple branches: http://i.imgur.com/WefAVGX.png. Notice the branches 'official' and 'stable'. 'Official' is just the various releases of arc, and 'stable' is releases + small bugfixes. To see what's changed in 'stable' since arc3.1, look at the list of commits: https://github.com/arclanguage/anarki/commits/stable. There's only 15 commits or so on that page after the arc3.1 release, and you can click on them to see what was changed. Scanning the list, really there's no other important bugfixes.
I keep a directory called arc3.1 by doing:
$ git clone http://github.com/arclanguage/anarki arc3.1
$ cd arc3.1
$ git branch -a # list all branches
$ git checkout stable
I can then see the same list of commits by typing 'git log'. Or generate diffs by typing:
$ git diff remotes/origin/official
This is enough information for you to create and manage a repo just how you described. I have a slightly different itch, so I have a different approach: http://github.com/akkartik/arc. Back before me, a user called aw created a new approach as well: http://awwx.ws/hackinator, which I doubt is actively maintained. You should create one too and show us. If it's substantially better we'll switch to it. Over time.
I'm sure everything could be much better. (I'm trying to create a new way to organize programs, after all: http://akkartik.name/about.) But no matter how you organize programs, people would still need to know about some baseline tools, like diff and patch and scripts in your description above. For anarki you need to learn some git at the moment.
If you're going to resort to patches to manage code as text, that's what source control systems like Git are good for. They also manage the emergent complexities with this approach:
- They help highlight one version of the code as the canonical one that most people are expected to be using. That way development effort isn't wasted on out-of-date code combinations that nobody cares about.
- They provide ways to merge textual changes, revert them, and otherwise apply them out of order.
What you have in mind might be a coherent alternative to the existing version control systems, so I'm not just saying to give up. :)
---
"Is there a way to get a diff?"
Git makes it easy to create diffs (using the "git diff" command with various arguments), and GitHub makes it particularly easy to view certain kinds diffs on the Web. In GitHub, if you're on the stable branch and you click the "commits" button you get to this page:
Don't bother trying to copy and paste a diff from GitHub though. I think you'd have better luck at a command line, even if it means learning several commands. (I'm not sure I know all the right commands myself though. >.> )
---
"If it's Arc 3.1 why is it named Anarki?"
There are a lot of ways to answer that question. If someone wants to make a contribution to Arc so that every other Arc user can benefit, that's what Anarki is for. However, not everyone uses Anarki, so the fantasy is that it's Arc, but the reality is that it's a specific project that we refer to by a specific name.
I'm not actually sure why that branch of Anarki exists, but it looks like it was created to support plain Arc 3.1 coding without all the extensions people made to the master branch. Its differences mostly have to do with adding text editor extensions and shell commands to support Arc 3.1 programs.
"Don't bother trying to copy and paste a diff from GitHub though. I think you'd have better luck at a command line, even if it means learning several commands. (I'm not sure I know all the right commands myself though. >.> )"
True. I coulda sworn github had a raw view. Was that just for snapshots?
Thank you so much for this!! arc-js looks great. I'm able to define and run Arc macros with it.
This changes everything. With arc-js you can start writing client-side apps in the browser without dealing with Javascript. You can add libraries in arc-js that do that (are there any already?) The big advantage is you bypass HTML: you can make a browser and a server send sexps back and forth.
There I was thinking there was no hope with an Arc in Javascript and sure enough Arc is quietly thriving.
When I looked at wat.js it didn't have macros. That was a bigger drawback than speed, because then you can't generate any HTML. An Arc in Javascript could get away with implementing less, like ignoring continuations and garbage collection.
I'm pretty sure it has macros. Even if it didn't, their effects are easily duplicated with fexprs.
Rather, a macro can be described as an extension of the compiler, while a fexpr is an extension of the interpreter. In a compiled language, macros can be an optimization tool, because they can perform arbitrary computation at compile time.
In wat.js, which I believe is interpreted, macros provide relatively little value. They can still perform slightly better, because they only need to be expanded once. To quote directly from manuel:
"To fexprs I also add macros. When a macro is used as the operator of a form, the form's code gets changed to the macro's output when the macro is first called, a technique I learned from here. I like macros because they make syntactic abstraction cost-free - with fexprs alone there is always an interpretative overhead. Still, Wat macros, like fexprs, do not work with quoted identifiers, but with first-class values, so many hygiene problems are avoided."
So generating HTML with the same kind of dsl that arc uses should be fairly painless. Either that, or I misunderstood what you were saying entirely.
I must have missed the macros then. The biggest problem I had with wat-js is that... I couldn't figure out how to run it. I wish it had a single shell script that you ran and brought up a repl. So one doesn't have to spent time copy-pasting stuff from a text file into an html file and firing up a browser, or typing their code within double quotes or doing anything else.
I tried them now and was able to get up a Wat VM. But I haven't been able to figure out how to run macros in there.
I wish there were examples. Snippets you can copy paste that do lots of useful stuff like http://ycombinator.com/arc/tut.txt. It shouldn't take anyone more than 10secs to find how to do things. It's disheartening to hear such good work remains unused only because there are no examples but I suspect it's the truth. That's what happened to me.
Language designers think some cool esoteric feature is the killer feature of a language but a more useful feature might be examples: an implementation of an application developers want, like Arc did by providing a webserver and a blog.arc. If you are writing a language, ship a useful app with it, like C did with UNIX.
I would look at the boot.js file. It's kind of like arc.arc is for arc, in that it is a lisp-based bootstrap of all of the basic functions made available in the default environment.
It should give you basic examples of how to write code in wat, as well as show you which functions are defined. Of course, most of them don't actually have examples in the file, as it's merely defining the standard library, but it's a start. There's also test.wat, and the authors blog: http://axisofeval.blogspot.com/search/label/wat (which unfortunately mostly uses the underlying json representation for most of its code).
And the 'killer feature' of wat.js is not that it has first class environments or fexpr, which are admittedly very cool, but that the core language implementation that supports all of these and more is only a few hundred lines of code. There's more in the repository of course, but all of that is to provide the standard library or add sexpression parsing, etc. This makes wat a very good minimal platform for building a custom language.
Are you fully sure there is no other way to do what you want to do without it taking 20 years?
That's a question worth pondering, not just because big things tend to fail or good design is simple, but also because life has a way of changing what people choose to do over time.
Yeah, I'm actually constantly looking for shortcuts, ways to not need to reinvent the stack, ways to build things that will help existing projects, or allow people to use existing languages and tools.
When I said 20 years I wasn't saying I have a plan that's going to take 20 years. I meant that I'm building small projects that -- when I step back -- seem to be headed in that general direction over time. Projects like wart[1], tracing tests[2] and layers[3]. I'm building these little projects out in the open, so hopefully I won't fall victim to the "hacker comes out of basement after 20 years with a compiler and OS that nobody cares about" failure mode. No, my goal is to fail in some more creative way :)
I'm less worried about my goals changing. On the one hand, if they do they do, one can't be anything but fatalistic about it. On the other hand the goals I outlined above seem to have hooked into my motivation at a deep level. I may well end up switching solutions I explore, but it seems like the goal is unlikely to change.
I haven't really started building anything for the past couple of months, but I've been thinking lately about three potential little projects: a) a simple multitasking OS without virtual memory or processes, maybe even without preemptive scheduling, b) a better language for teaching programming by separating scope from the notion of functions, and c) an assembly language with structured programming and lisp macros somehow. Something 'easy' to build without needing an entire C compiler. Rephrased: "I'm willing to do my own register allocation. What can you do for me, lisp?" :)
Mostly I just get up every morning and work on whatever I want. And these are the things I've been ending up working on.
c) let doesn't work with more than one variable; so withs is more general (should let even exist?). I can't tell what the difference between with and withs is, and I had picked the latter when I started needing more than one variable. What is their difference?
Yes, let is just a simpler form of withs, but it's worth using because it signals that you only need one var and therefore skips extra parens.
withs is the sequential form of with, where you'd like each variable available in defining later variables within the with. You have to say:
arc> (withs (x 1 y (* x 2)) (+ x y))
..but you can use the simpler with in:
arc> (with (x 1 y 2) (+ x y))
Not only can you, you should. Using the most general form when a simpler form will do is akin to crying wolf; it's very useful when reading code to be able to tell simpler parts from more complex parts at a single glance.
So use with to signal that you're defining multiple variables at once. Use withs to signal that there's a dependency between the definitions. If there's no dependency, with is more idiomatic. And if there's only one variable, let is more idiomatic. Finally, a withs with multiple bodies and a trivial body is easier to read if you put the final expressions into the body. This:
(withs (x 1
y (f x z))
y)
is better written as:
(withs (x 1)
(f x z))
and therefore:
(let x 1
(f x z))
(and further to (f 1 z), but let's pretend x is being bound to something less trivial.)
Even if a body is a little more complex:
(withs (x complex-expr1
y (complex-expr2 x)
z (complex-expr3 y))
(car z))
it's sometimes nicer to read as:
(withs (x complex-expr1
y (complex-expr2 x))
(car (complex-expr3 y)))
let, with and withs are nothing but macros, so if you avoid using them you're giving up the benefits of lisp. Might as well go back to a weaker language. In general, my idea when learning lisp was: "Since I come from an imperative background my tendency is to define intermediate variables. Therefore I will start by avoiding all temporaries, and only after I get it working will I insert the fewest possible temporaries to make the code readable." It's stood me in good stead.
Thank you for writing up such a clear explanation.
There are definitely safety benefits to using the simplest form. But I would prefer to use the most general form (withs) because I won't need to worry about using the most idiomatic form or to switch from one form to another as the program changes. It frees me to worry about other things.
Being able to tell simpler parts from more complex parts at a single glance isn't as much of a concern for me. I noticed I don't go back to even read code of programs that didn't solve a big enough problem. The programs reach a dead end and don't develop further. There are more important things to worry about when writing a program.
Also note what let, with, and withs offer could be provided in other languages too. Them being implemented as macros in Arc is incidental. You're not giving up the benefits of lisp by not using them.