This seems like egregious hype mongering. Two examples:
a) "Pure functions are easily parallelizable since they encourage immutable data structures which reduce the side-effects that make code hard to run on multiple processors. This is how Bitcoin will reach its infinite scalability."
Pretty abrupt transition there from a sentence that had me nodding along to one that gives me whiplash. Wait, wha? Pure functions are used in lots of places outside Bitcoin. None of them brags infinite scalability. So perhaps you need more than pure functions?
b) "@TensorFlow uses the functional programming paradigm of lazy evaluation. A tensor flow graph exists separately to the computation of that graph.
"Bitcoin can and will be used to create true artificial intelligence."
I don't think I need to say anything about the last sentence. For the rest, the author needs to make up their mind whether the paradigm is functional programming or lazy evaluation. Lazy eval is just a mechanism. Tensorflow uses it in the part of the system that is stateless. Other parts of ML are incredibly (and incredibly subtly) stateful.
My homebrew blog server has a part with lazy evaluation. It's a few lines of code. It's not going to give me infinite scalability anytime soon.
 Since many ML systems don't explain their models, there's a state dependency from any results they provide to all training data they've ever seen.
I'm not following the two of you on precisely what this anti-feature is. Assigning to local variables using '=' does not create new global variables. Can you share a code sample showing what you're talking about?
Sorry I just saw this. Can you elaborate? What is an example of a variable that C and Python define correctly but has a worse equivalent in Arc? Or what's a variable in Arc that bit you by being globally scoped?
In particular I'm surprised that you mentioned C. If we're worse than C in this respect I'd love to understand that better.
He wants it to be a little more like scheme wherein:
(define v 10)
creates v as a local variable. In Arc, and most lisps, you need to do:
(defun f ()
(let ((v 10))
Additionally, Arc has the wrinkle (i think, i haven't used Arc recently) where if you do:
(defun f ()
(= v 10)
you create a global variable and assign 10 to it, instead of the python default which creates v in local scope. Basically, he's trying to use imperative programming and doesn't want to declare local variables using let.
I see. Yes, it can be a little sharp-edged for new-comers from other languages that `=` auto-defines globals. It's not immediately obvious that we don't use it much. Let me think about how to improve that.
Yeah, as a newcomer I can attest that I did not know that. V here being global is really counter-intuitive:
(defun f ()
(= v 10)
I would have expected (and would prefer) the default to be to bind to whatever the current scope is, and to have global (file level, then application level) scope be opt-in rather than opt-out. We can't assume that it's more likely new users will be familiar with lisp idioms.
In your example, the variables in scope inside `f` are exactly the same as the variables in scope outside of it. You're implying the "current scope" has changed, but if the set of variables in scope hasn't changed, then what other part has?
I think you were expecting a concept that Arc doesn't have. Adding an unnecessary concept to Arc would make the language more complicated for newcomers who weren't expecting it, right?
Of course, if users do consistently come in with the same intuition, it's pointless to design against it. Often we've gotta design for complex realities rather than simple principles. :) If you think it's a case like that, I can sympathize, and maybe someday I'll see it.
Sometimes I feel like general-purpose plain text programming language is such a specific topic that it leads to only one possible language design. Feeling that way is probably the only way I'll design a single language at all, rather than designing a lot of half-languages and never finishing any of them!
In this case, I actually _don't_ feel like there's _no_ potential to alternative notions of lexical scope or variable assignment, but I think Arc's at a sweet spot, and I've some extensive reasoning as to why....
"We can't assume that it's more likely new users will be familiar with lisp idioms."
If not some other language's idioms, where did you get the idea of there being a "current scope"?
It's true that many popular languages have features where they infer a variable declaration at some notion of "current scope" around innermost point (Python) or outermost point (CoffeeScript, Ruby, MATLAB) where a variable is assigned. Newcomers to Arc from to those languages might expect this. (I think uu must be bringing in Python experience.)
I think some languages (R, Kernel, maybe some Scheme interpreters) represent the lexical scope as a run time data structure, and variable assignments can add new variables to the local scope that were previously looked up from an outer scope. They have lexical scope, but arguably not static scope.
I also want to mention PHP, which is off doing its own thing where there's hardly any implicit inheritance between lexical scopes at all. Every variable lookup or assignment is restricted to the current function unless there's an explicit `use` or `global` declaration to imply otherwise. I kind of admire PHP's willingness to make the interaction between scopes explicit like this; it means PHP could evolve to have different parts of the code written in different languages, with explicit marshalling of values between all of them.
So let's look at Arc as its own language.
Thanks to Arc's lexically scoped `fn`, it's basically an extension of the lambda calculus, and it has easy access to all known lambda calculus techniques for Turing-complete computation. This means Arc programmers basically don't have to use assignment at all unless they want to.
In Python, those lambda calculus techniques are possible to use in theory, but every nontrivial lambda must be named and pulled out onto its own line, giving us something a lot like `goto` label spaghetti.
In PHP, every nontrivial lambda must have a `use` declaration to pull in all the variables it captures. This can get to be particularly verbose, eventually to the point where it might be easier to pass around explicit context objects.
Even using lambda calculus techniques a little bit in Python or PHP means we start to have trouble with mutable variables. Lambda calculus uses functions for control flow, but using functions in Python or PHP means creating new scopes, which means we can't easily assign to outer variables from inside our conditionals and loops. Most uses of mutable variables involve some kind of conditional or loop (or variable capture for its own sake), since that's what makes them anything more than a sequence of variables that happen to share the same name. So the more we use lambda-calculus-style conditionals and loops, the less we effectively have access to mutable variables in the programs we're writing.
In both Python and PHP, it just takes a little more boilerplate to work around this: We give up on mutable variables altogether and simulate them with immutable variables that refer to mutable objects. (There's also `use (&$foo)` in PHP and `nonlocal foo` in Python, if you prefer not to give up on mutable variables, but they amount to far more boilerplate.)
The standard boilerplate for these things in Arc is pretty much less than zero, thanks to macros. An Arc programmer can write a custom conditional operator as a higher-order function, and then when they're tired of putting the conditional branches in `fn` every time they use it, they can write a macro that generates the `fn` automatically.
Since Scheme and Common Lisp were already well-worn combinations of lexically scoped `lambda`, mutation (`set!`/`setq`), and macros, all of this could pretty much be predicted when Arc was designed.
Nevertheless (or maybe out of having different goals than I'm expressing here), Paul Graham and co. tried out automatic local variables anyway. It was implemented for an early, unreleased version of Arc. Then they pulled this feature out because they realized they kept introducing or removing lexical contours by mistake and breaking parts of their code. I bet this is because they were implementing some of their control flow macros in terms of `fn`.
Could it be possible to follow through on their experiment without recognizing all the same mistakes and pulling the plug again? Yes, I bet it is.
But I think Arc's local variable scoping rules and variable assignment behavior are exactly what they need to be:
- Implicit inheritance of lexical scope to enable lambda calculus techniques (unlike PHP).
- The easy ability to mutate variables in outer scopes so mutation can work together with lambda-calculus-style control flow (unlike Python, R, and Kernel).
This still leaves the CoffeeScript/Ruby/MATLAB approach on the table, where only the outermost assignment is treated as a declaration. I don't particularly like this approach, and that's because I prefer for the outermost level to be relatively seamless with the rest of the language. That way it's easier to break parts of the language off into optional libraries when it turns out they're not as helpful as expected. Arc's top level already isn't seamless with the rest of the language, but I think this would be a step in the wrong direction.
In summary: If users come in with incorrect ideas about Arc's variable assignment behavior based on their experience with other languages, I think that's most likely a place where other languages could learn something from Arc rather than the other way around. The higher-order techniques of lambda calculus are a sweet spot in language design, and Arc's system for local variable scope is well tailored to that. The Arc designers originally did try automatic local variables. They found them to be unnecessarily complex to work with, and I agree.
 http://www.paulgraham.com/arcll1.html "Here is a big difference between Arc and previous Lisps: local variables can be created implicitly by assigning them a value. If you do an assignment to a variable that doesn't already exist, you thereby create a lexical variable that lasts for the rest of the block. (Yes, we know this will make the code hard to compile, but we're going to try.)"
 http://paulgraham.com/arclessons.html "In Arc we were planning to let users declare local variables implicitly, just by assigning values to them. This turns out not to work, and the problem comes from an unforeseen quarter: macros. [...] In a language with implicit local variables and macros, you're always tripping over unexpected lexical contours. You don't want to create new lexical contours without announcing it. [...] It seemed to us a bad idea to have a feature so fragile that its own implementors couldn't use it properly. So no more implicit local variables."
 In Racket, the `racket/splicing` module (https://docs.racket-lang.org/reference/splicing.html) has a few rough edges, but it's a good example of how the choice of whether a macro changes the "current scope" can be controlled deliberately, even in a language with lambdas and macros. I didn't bring up Racket or Scheme's notion of "current scope" with all the other examples because it doesn't interact with variable assignment, but I think even that notion is a kind of ill-conceived complexity that I'm glad Arc doesn't have. It's handy to have local syntax that roughly resembles the top level to aid in refactoring, but on the one hand the resemblance isn't required to be perfect (and isn't perfect in Racket), and on the other hand the Scheme top level isn't very easy to make modular, so it's not even a good thing to resemble.
I have been thinking about a modified version of LET that takes a sequence of expressions (like begin) that are either assignments or not assignments. Then it would collect all consecutive assignments together into a letrec. For example:
(= a 1)
(= b 2)
(= c 3)
(= x 4)
(= y 5)
Sure, but losing data you don't want to lose because you reloaded a source code file does seem like more of an architectural than language issue. It would be a code smell in any other language.
My comment was slightly facetious but the more I think about it the more I'm wondering whether something like redis or php's apc wouldn't be a good idea - and not just as a lib file but integrated into Racket's processes for dealing with arc data directly.
It could serve both as a global data store and a basis for namespacing code in the future (see my other rambling comment about namespaces), since a "namespace" could just be a table key internally.
Otherwise hitching the code to a third-party db, as a requirement, would really limit what could be done with the language and would create all kinds of problems. You would be locked into the db platform as a hardened limitation. You would inherit the complexity of external forces (i.e. what if some other app deletes or messes with the db). What about securing the access/ports on the db.. etc..
It's always possible, but I think you would have to implement something internal where you can properly isolate and fully support all platforms the language does.
Seems likes namespaces would solve these problems the right way.
Yes. Currently, the options we have for stateful data are file I/O, which doesn't work perfectly, or tables that can lose their state if the file they're in gets reloaded. I'm suggesting something like Redis or APC, but implemented in Arc at the language level, to separate that state from the source code.
I was also thinking (in vague, "sketch on the back of a coffee-stained napkin" detail) that it could also be used to flag variables for mutability and for namespacing. In that if you added "x" from foo.arc it would automatically be namespaced by filename and accessible elsewhere as "foo!x",so it wouldn't conflict with "x" in bar.arc.
>Otherwise hitching the code to a third-party db, as a requirement, would really limit what could be done with the language and would create all kinds of problems.
Yeah, but to be fair, Arc is already hitched to Racket, which appears to support SQL and SQLite, so maybe third party wrappers for that wouldn't be a bad idea as well... sometime in the future when things are organized enough that we can have a robust third party ecosystem.
Languages that have decent bindings to a database also have global variables that still have uses, and that can be lost when you restart the server or do other sorts of loading manipulations. There's a category of state that you want coupled to the state of the codebase.
Yes, you can definitely try to make these different categories of state less error-prone by architectural changes. But I don't think other languages do this well either. Mainstream languages, at least. I know there's research on transparent persistence where every global mutation is automatically persisted, and that's interesting. But I'm not aware of obvious and mature tooling ideas here that one can just bolt on to Arc.
All that said, database bindings would certainly be useful to bolt on to Arc.