Arc Forumnew | comments | leaders | submitlogin
A new anarki
6 points by rntz 5637 days ago | 11 comments
Anarki needs to be updated to arc3. I hope I am not too presumptuous when I say: Here's how I think we should do it.

CatDancer's "Sharing arc hacks" (http://catdancer.github.com/sharing-hacks.html) is good way of dealing with hacks to arc (though not as great for libraries -- http://arclanguage.org/item?id=9766) and should be an organizing principle for the new anarki. In brief, a hack is represented by a git commit which incorporates only the minimum changes from the raw arc3.tar to accomplish it. These hacks are tagged as "${author}.${hackname}.${release}" (EDIT: this is not entirely accurate, see CatDancer's comment, below). This allows people to grab precisely the hacks they want by using git's merging machinery.

Note that the tag name includes the author's name. This is a good idea for hacks which are clearly owned by and the responsibility of one person. However, for purposes of forward-porting hacks from anarki, I have used the name 'anarki' as a sort of synonym for the arc community. I suggest the use of this name in general for forward-porting hacks from anarki (unless you were the original author and wish to take ownership, of course), and in general if one wishes to give a hack up to communal management.

Isolating hacks from each other to produce a minimal change from arc3.tar requires either working on hacks in isolation - starting from raw arc3, sans even minor utilities like arc.sh or the exit-on-eof fix - or learning how to use `git rebase` and `git cherry-pick`. If you have the time, I strongly recommend the latter approach. It has been covered in detail elsewhere, but I direct you in particular to http://blog.moertel.com/articles/2007/12/10/how-i-stopped-missing-darcs-and-started-loving-git (unfortunately the git tutorial aspect is a bit mixed up with darcs-git comparisons), and for good style in using git-rebase, to http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html.

There will be four arc3-based branches in the new anarki: official, fixes, extras, and master. `official' tracks pg's releases of arc, as it does now. `fixes' contains only bugfixes to arc, such as the fix to make atomic-invoke work properly even with continuations. `extras' contains things which make living with arc easier, but which do not significantly alter the behavior of arc, like arc.sh and the exit-on-eof patch. In particular, any code that works in the extras branch should also work unaltered in the official branch (forward-compatibility), and of course vice-versa (backwards-compatibility). `master' should contain all code in fixes and extras as well as any enhancements people care to add, like anarki's old master branch.

Anarki's old stable and master branches (as well as my more-or-less defunct hygiene branch) will be renamed to arc2.stable and arc2.master respectively, so that anyone who wants to use them doesn't have to go hunting through `git log` to grab them. Unless the community (that's you guys!) think the above is a bad idea or have better suggestions, I'll make the switchover sometime this week. Until then, the fixes, extras, and master branch are available in anarki and named arc3.fixes, arc3.extras, and arc3.master respectively; feel free to check them out and contribute+merge hack-tags into them.

I'm still not sure what should be done regarding libraries. Uploading them to anarki in the lib/ directory was and is a simple way to share them, but various more featureful and distributed alternatives have been proposed - in particular CatDancer's "lib" library (http://catdancer.github.com/lib.html). Undoubtedly the community will work this out in time. For now, I suggest we focus on forward-porting things from the arc2-based anarki that are obviously bugfixes, extras, or language-additions.

On that note, there is also an arc3.help branch in anarki, in which I am working on forward-porting and improving anarki's on-line help system. I'd be glad for any help here, but remember not to bring in any unnecessary dependencies - in particular, do NOT merge arc3.master into arc3.help; you should only ever merge arc3.help into arc3.master. Note that, unlike the original help system, the documentation of core arc files (arc.arc, strings.arc, etc) is done in a separate file, docs.arc, rather than documenting them inline with help strings (don't worry, functions can still be documented inline; this just makes merging with other hacks much less of a pain in the ass).

The hack-tags which have been uploaded to anarki thus far, and some notes regarding them:

In arc3.master:

- anarki.help.0: A snapshot of the current (as of this writing) state of the arc3.help branch.

- anarki.util.0: Adds the util.arc file. Instead of adding functions to arc.arc, I propose to put them here instead. This helps keep arc.arc as close as possible to pg's arc3.tar, which helps when trying to understand what anarki actually changes about arc as compared with what it adds. Before adding to this file, however, please consider whether your needs might not better be served by adding a library. If it's truly a new language feature or something that needs to be autoloaded to be useful - like the help system, which is the only thing currently occupying util.arc - then and only then should it go here.

- anarki.load-file-stack.0: A change to arc.arc that makes 'load keep track of the current load file and the load file stack (as 'current-load-file* and 'load-file-stack* respectively).

- rntz.$.0: The '$ special form (http://arclanguage.org/item?id=9748), which permits access to mzscheme with a method of escaping back to arc.

- rntz.simplify-halve.0: A simplification of the implementation of 'halve in strings.arc.

- Everything in arc3.extras and arc3.fixes.

In arc3.extras:

- anarki.arcsh.0: Adds arc.sh.

- catdancer.exit-on-eof.1: The patch described at http://hacks.catdancer.ws/exit-on-eof.html. Makes arc exit on eof. I was unable to find this patch in CatDancer's arc repo, so I grabbed the patch itself, applied it, and made up a tag for it.

- anarki.gitignore.0: Adds a .gitignore containing the sole line `arc/', to make git ignore the arc/ directory which is generated by the arc server. Authored by nex3.

In arc3.fixes:

- anarki.fix-atomic-invoke.0: Fixes 'atomic-invoke to work even in the presence of continuations, by using 'dynamic-wind instead of 'protect.



4 points by CatDancer 5637 days ago | link

"${author}.${hackname}.${release}"

I recommend using publisher.hackname.release, where publisher is your Arc forum username.

For example, Linus Torvalds writes hardly a line of kernel code these days, he largely integrates the work of others. So there's a role for authors and a role for integrators.

Under my hack naming scheme, all the hacks you publish would be rntz.hackname.release. The hack may have one author or multiple authors, you may not be the author or may not be one of the authors.

"catdancer.exit-on-eof.1" is an exception, because it already has a name, and you're really just importing the hack into git, rather than doing your own integration. (If, however, you changed it to get it to work with Anarki, then you'd use "rntz.exit-on-eof.0").

The reason for using the publisher in the hack name is to make it globally unique. For example, you and I may both happen to port "arcsh" to arc3. For people who aren't using git, it's useful to be able to refer to hacks and download them by a unique name. By using "rntz.arcsh.0" and "catdancer.arcsh.0" there's no conflict. It should be understood that the presence of the publisher's username "catdancer" or "rntz" in the name does not imply authorship.

-----

2 points by Adlai 5637 days ago | link

Should there be a periodical (i.e. once/twice a month) effort to merge together various publishings of the same hack? I think it's important to do that, so that we don't have five differently-named but almost identical versions of a hack floating around after a few people add features. A list of contributers to a hack/library can be kept as commented "meta-data" within the file, and the hack can then be renamed to anarki.hackname.release.

-----

3 points by shader 5637 days ago | link

You are welcome to do merges whenever you like ;)

However, I will point out that the name of the project/community is "Anarki", and so it is very likely to be distributed and each person will probably want their own version anyway.

-----

2 points by CatDancer 5637 days ago | link

Just like Linus Torvalds decided he wanted to be the Linux integrator, someone will emerge who wants to put together the best version of various variations of various hacks. If he or she does a good job, he or she will become respected and people will look first to him or her to find a version of a hack they want to use.

Information about a hack, like "this hack is the best of the bunch", is really best not put into the hack name, because later if the information changes (maybe you find out the hack has a bug), now you have a name that's giving bad information.

-----

2 points by CatDancer 5637 days ago | link

I'm still not sure what should be done regarding libraries

There are some advantages to keeping each library in its own branch:

- It makes the git development history for the library really clear.

- I can propose changes to a library and merge them in to the library branch, without getting messed up by independent merges happening to other libraries.

- If you're using library A and someone commits a change to library B which breaks A, but you can't go back because someone later committed C which you need, what do you do? Make a commit that reverts the B change? But what if someone else is using that B change? Separate branches avoids these kinds of conflicts.

But, of course keeping libraries in separate branches does make using the libraries harder. Instead of just checking out one branch, now you need to use the published versions of the libraries, or checkout a git repository for each library.

I do no longer recommend my idea of having the git commits for a library include their dependencies. That was a good experiment for me to try, and I learned a lot from it, but it turned out not to be useful.

If you do decide to give each library its own branch, then I recommend that the library branch not include the Arc sources: make it an independent branch. The easiest way to do this is to add the library to a new, fresh git repository, and then fetch the library commit into your Anarki repository as a branch.

-----

1 point by shader 5637 days ago | link

I also agree with separate branches for each library. For one thing, it takes less effort to make the branch stand alone, and there is less chance of messing things up.

Also, they don't get in the way as you are not forced to read the list, and can choose to track as many or few remote branches as they like.

How about using one branch per library, and tags for versions? That way everyone can track the development and even add things, but the versions are the things that 'lib and others like it go for.

The only thing I'm wondering about branches for libraries is: how do you find them if you're trying to fetch them via 'lib? I think only major changes need to be in their own branches, like the hygienic macro system. Everything else should either be hosted separately or in the lib folder.

-----

1 point by CatDancer 5637 days ago | link

The only thing I'm wondering about branches for libraries is: how do you find them if you're trying to fetch them via 'lib?

I can think of three scenarios:

1. You're using the public, published version of a library that also happens to be in your git repository.

2. You're using an unpublished version of a library in your git repository: for example, you're testing your code against the latest development version of a library to see if they work together.

3. You're working on a library.

For #3, I presume you'd just be using 'load.

For #2, if you want to for example get the latest version of a file "foo.arc" in the "foo" branch, you can use git for that, e.g.:

  git show foo:foo.arc
For #1, you don't need to get the library out of git, you can use whatever mechanism we're all using to fetch and use a published library.

-----

1 point by shader 5637 days ago | link

For #3, I presume you'd just be using 'load.

I was hoping to use a modified version of 'lib that would allow me to type

  (lib binary)
when I meant

  (load "lib/binary.arc")
It has the advantage of less typing, and hopefully the concept that functions are not reloaded unless the file has changed. If I'm trying to test a new version of a function, I would rather selectively reload that function than the whole library.

-----

1 point by shader 5635 days ago | link

What should I do in git if I have been developing a branch off of master, and I want to move it over to be based on arc3.master so that it will be more forwards compatible?

I haven't changed anything that existed in arc3.master, or master for that matter, just added some new things, and modified them a bit.

I thought I was supposed to use "git rebase arc3.master" while in the other branch, but it seems that didn't work, as it was complaining about merging changes in arc.arc, and I never touch that file in this branch.

Suggestions?

-----

1 point by rntz 5635 days ago | link

Well, first of all, `git rebase --abort` if you haven't already to avoid fucking things up. The actual command you want is `git rebase --onto arc3.master master ${my-branch}`. Explanation follows.

`git-rebase $target $branch` takes the changes made by the commits in $branch relative to its nearest ancestor in $target and attempts to change them to be relative to $target - it "forward-ports" the changes you've made relative to an old version of $target. The problem in your case was that the common ancestor of arc3.master and master is plain old arc2. So what it did was attempt to take essentially the whole of anarki's changes to arc2 (plus whatever changes you were working on) and forward-port it to arc3.master. Something of this magnitude is not really amenable to handling by automatic merge. Conflicts are inevitable, and moreover, a semantic change (like the rename of 'set to 'assign and 'assert to 'set) won't be caught... there's a reason that the arc3 anarki is a fresh start instead of a straightforward port.

`git-rebase --onto $newbase $oldbase $branch`, however, takes the changes made by the commits in $branch relative to $oldbase and tries to forward-port them to $newbase, rather than just taking the nearest shared ancestor. `git-rebase --onto arc3.master master $branch`, therefore, says "take the changes between $branch and master and forward-port them to arc3.master", which is a much more feasible thing to do automatically if those changes haven't touched anything that differs significantly between master and arc3.master.

-----

1 point by shader 5635 days ago | link

Thanks.

I already did the git rebase --abort as soon as it started complaining about something I new I didn't change.

What if master has changed since I started the branch? Should the command instead be 'git rebase --onto arc3.master mybranch~n mybranch'? Or does it not matter where the head of master is relative to the beginning of my branch? Should I rebase to the head of master first, and then move it over to arc3.master?

I guess the documentation on git rebase wasn't quite detailed enough for me ;)

-----