NHacker Next
login
▲My Ideal Array Languageashermancinelli.com
148 points by bobajeff 34 days ago | 71 comments
Loading comments...
adregan 34 days ago [-]
The author of this post was the guest on the most recent episode of the podcast The Array Cast

https://www.arraycast.com/episodes/episode111-ideal-array-la...

npalli 34 days ago [-]
Strange to read this article and find no mention of Julia (but APL, Mojo, MLIR BQN etc.. which are not exactly widely used languages). It checks many of the boxes

User-Extensible Rank Polymorphism is just beautiful with the broadcast dot syntax. I don't think any other language has this clean and flexible implementation.

Others -- GPU programing, parallelism, etc. are pretty good with Julia. Real shame it hasn't taken off.

electroly 34 days ago [-]
I'm in a position of supporting a Julia environment but not writing Julia myself. From my perspective, they need to fix time-to-first-plot before it can be adopted more broadly. It's horrendous. The "2-3 minutes" I see online is an aggressive estimate; it's more than that for our modest set of data science packages on very beefy workstations. I had to use PackageCompiler.jl to build a sysimage which shifted a ton of burden onto me (and onto GitHub Actions) to avoid a long precompile on every user machine. I had to do the same to get my Julia Docker image to stop precompiling on every new cloud machine even though it had already been precompiled during docker build. I would describe this process as a nightmare, and it was a serious problem--the thing was precompiling every time on every job run in the cloud using the docker image.
xgdgsc 34 days ago [-]
That' s solved more than 1 year ago https://news.ycombinator.com/item?id=38783886 https://modernjuliaworkflows.org/optimizing/#compilation .
aragilar 33 days ago [-]
Are you actually sure it's solved? I note that Julia proponents have a history of claiming that an issue has been solved when it is apparent that it has not been solved (see e.g. https://danluu.com/julialang/).
electroly 33 days ago [-]
The events of my post happened in the last two months on Julia version 1.11.5, and this poster simply linked to a primer on PackageCompiler.jl which is the thing I said I was using in my post. It's not solved; at best there are complicated mitigations that let you control when the long compilation happens without avoiding it. If you try out Julia, you will run into the issue, and as a first-time user you won't be prepared to implement the mitigations, none of which are suitable for ad hoc usage. The sysimage only works for us because we rarely change our packages so we can build it ahead of time in GitHub Actions.
xgdgsc 33 days ago [-]
The tradeoff between compilation time and programmer productivity and running speed is worth it compared with using any other language. You should consider most users don' t need to build an image . They just do things in the REPL with Revise.jl (which is fast). Or use a bit of PrecompileTools.jl to get good enough startup time (which isn' t complicated if you are the code writer). Package compiling works even large and slow is a bonus. https://pypl.github.io/PYPL.html Julia still grows despite all those controversial articles for a reason.
xgdgsc 33 days ago [-]
Yes. I don' t know why you' d rather trust an article from 10 years ago to understand current status.
pjmlp 33 days ago [-]
It has on some industries, keep it steady and slow, eventually more people will notice it.

https://juliahub.com/industries/case-studies

Many languages could dream to have even a third of such high profile case studies.

wolvesechoes 33 days ago [-]
Shame that it had so many nice ideas, but everything about Julia is half-baked at best.

I like to play with it, but just it.

slwvx 34 days ago [-]
> User-Extensible Rank Polymorphism is just beautiful with the broadcast dot syntax ...

Just to be clear, I guess that Julia's broadcast (dot) syntax is an implementation of "User-Extensible Rank Polymorphism"; is that right? Or does Julia's dot syntax include more than UERP?

capyba 34 days ago [-]
I love working in Julia, it makes clean numerical code so easy to write.
mighmi 34 days ago [-]
Julia may be cool, but it's not an array language in the tradition of APL.
v9v 34 days ago [-]
I've understood array language to mean "a language with focus on array processing" rather than "APL descendant". The array language comparison I found online lists Fortran, MATLAB, Julia etc. as array languages: https://github.com/codereport/array-language-comparisons
cl3misch 34 days ago [-]
I think none of the languages in the linked article are, and that's not what it's about as sibling comments point out.
oivey 34 days ago [-]
Sure but neither is Mojo.
34 days ago [-]
abcd_f 34 days ago [-]
> User-Extensible Rank Polymorphism

> IMO this is what makes something an array language.

Great to hear. So what is it?

preommr 34 days ago [-]
Not op, but I assume it means that there's rank polymorphism (i.e. data can be of arbitrary dimensions, and there's support for things like functions working on both N-dimensions, without having to specify n, or maybe setting constraints on n), and that the polymorphism can be used on the programmer side (so it's not limited to just a handful of language builtins) through the oop equivalent of subclasses and interfaces.
goldenCeasar 34 days ago [-]
A question, would you interpret this as rank polymorphism?

  schema do
    input do
      array :regions do
        float :tax_rate
        array :offices do
          float :col_adjustment
          array :employees do
            float :salary
            float :rating
          end
        end
      end
    end

    trait :high_performer, input.regions.offices.employees.rating > 4.5
    
    value :bonus do
      on high_performer, input.regions.offices.employees.salary * 0.25
      base input.regions.offices.employees.salary * 0.10
    end
    
    value :final_pay,
      (input.regions.offices.employees.salary + bonus * input.regions.offices.col_adjustment) *
      (1 - input.regions.tax_rate)
  end

  result = schema.from(nested_data)[:final_pay]
  # => [[[91_000, 63_700], [58_500]], [[71_225]]]
CraigJPerry 34 days ago [-]
I think i'm misunderstanding, rank is explicit throughout this example but i'm not familiar with this syntax (ruby maybe?) but whatever the case i don't see the rank polymorphism.

If i'm reading the syntax correctly, this would translate in kdb/q to a raze (flatten) on the 3 dimensions (regions, offices, employees). Probably more likely to be expressed as a converge but in either case, the calculations here are not possible in a rank polymorphic way.

goldenCeasar 34 days ago [-]
The broadcasting handles the rank differences automatically. When bonus (at employee level) multiplies with col_adjustment (at office level), each employee's bonus gets their office's adjustment applied, no flattening or manual reshaping. The structure [[[91_000, 63_700], [58_500]], [[71_225]]] was preserved.

This is from a Ruby DSL I'm working on (Kumi). Probably the broadcasting semantics are very different from traditional rank operators in q/APL?

Edit: I realized that I missed the input structure:

  Region 0: Office 0 has 2 employees, Office 1 has 1 employee
  Region 1: Office 0 has 1 employee
CraigJPerry 34 days ago [-]
Just had a quick browse of the kumi readme - very cool, i like it
goldenCeasar 33 days ago [-]
That means a lot, thanks!
npalli 34 days ago [-]
I didn't downvote but was utterly puzzled with your example. After your response below, it occurs to me that you are confusing Employee rank (a business concept) with Array rank a mathematical concept. Either that or it is very strange explanation for rank polymorphism.
djoldman 34 days ago [-]
The programmer can define functions that operate on matrices without having to be explicit about the number of dimensions and possibly (types of data, size of data, or length).

Example 1: A function that can take as input a 4x2x8 matrix or a 3x7 matrix.

Example 2: A function that can take as input a 4x2x8 matrix and a 3x7 matrix and output a third matrix.

tomsmeding 34 days ago [-]
Rank polymorphism means that a function can be polymorphic in the additional dimensions of arrays. For example, if you write a function that takes a 2x3 and a 4x5 array, it can also work on 10x15x2x3 and 10x15x4x5 arrays by broadcasting.

If rank polymorphism results in accepting both 4x2x8 and 3x7, then that means the function was a function on elements to begin with. Which is possible, but not the most interesting application of rank polymorphism.

djoldman 34 days ago [-]
> Rank polymorphism means that a function can be polymorphic in the additional dimensions of arrays. For example, if you write a function that takes a 2x3 and a 4x5 array, it can also work on 10x15x2x3 and 10x15x4x5 arrays by broadcasting.

Thanks, this is what I was ineloquently attempting to describe with "A function that can take as input a 4x2x8 matrix or a 3x7 matrix."

almostgotcaught 34 days ago [-]
> A function that can take as input a 4x2x8 matrix and a 3x7 matrix and output a third matrix.

which shows that this feature request is complete jibberish

rscho 34 days ago [-]
Why gibberish ? It's a common feature in both array languages and Iverson ghosts, and many find it extremely useful.
tracker1 34 days ago [-]
You mean like a "winner" function able to check for both Tic-Tac-Toe, a Connect Four field and a Similar 3D+ tower game?
taeric 34 days ago [-]
I'm not sure I follow? The "winning" condition is different in all of those examples?
tracker1 34 days ago [-]
X adjacent cell values in an N dimensional array? For tic tac toe, it's 3 in a row, for connect 4 it's 4 in a row.
taeric 33 days ago [-]
Fair enough, I guess. The "available moves" would be dramatically different, such that I'm not clear I see much of a win on this reuse. But, yeah, fair enough.
tracker1 33 days ago [-]
It was just the simplest case I could think of that would be reasonable. I'm not a heavy enough pure math guy to really get how an Array centric language is even all that valuable myself TBF.
CapsAdmin 34 days ago [-]
game math libraries often have this (and glsl gpu shader language), like "2 * vec3(1,2,3)" results in "vec3(2,4,6)"

There are other cases like adding vectors to matrices and so on, but in the end this logic is defined in some custom add operator overload on a class or object in the language.

(I had no idea what it meant either until i searched for examples..)

cl3misch 34 days ago [-]
I wondered the same. Similarly I wish the author had provided examples for statements like

> Numpy also needs to be paired with a JIT compiler to make python a real array language

34 days ago [-]
pklausler 34 days ago [-]
My ideal array language is one in which array operations are function compositions, since arrays are functions. A functional view of array expressions naturally minimizes needless temporaries in most cases.

See https://github.com/llvm/llvm-project/blob/main/flang/docs/Ar....

goldenCeasar 34 days ago [-]
Funny, on another totally unrelated domain (business logic/rules engines) I was building something very very related - array broadcasting with semantic preservation through arbitrary nesting levels
rramadass 34 days ago [-]
A previous relevant discussion since there is so little on Array Languages - https://news.ycombinator.com/item?id=38981639
hinkley 34 days ago [-]
You explain the evolution of CPUs but then don’t explain Rank Polymorphism.
jph00 34 days ago [-]
"Rank" means the number of dimensions of an array.

So "rank polymorphism" means being able to write expressions that work correctly regardless of how many dims the arrays involved have.

For example, in numpy you can write a function that handles both lists and matrices automatically, by taking advantage of broadcasting. (The J language takes this idea a lot further -- numpy is a fairly minimal implementation of the idea.)

hinkley 34 days ago [-]
That just sounds like someone needing to feel smarter than 'multidimensional arrays' sounds. Which if you ask the average unskilled laborer, already sounds pretty damned fancy.
diziet_sma 34 days ago [-]
It's not the same thing as multidimensional arrays though. You can have multidimensional arrays without rank polymorphism. Rank polymorphism makes working with multidimensional arrays much easier because you can write one function which works over input arrays with different shapes.
IncreasePosts 34 days ago [-]
It's just like polymorphism, only stinkier
34 days ago [-]
nromiun 34 days ago [-]

  ⊢×0≠∧˝˘∧⌜∧˝           # Marshall & Dzaima (tacit!)
  (≠⥊∧´)˘{×(⌾⍉∧)0≠} # Dzaima & Rampoina
  {×(∧˝˘∧≢⥊∧˝)0≠}     # Dzaima
Call me old fashioned and stuck in C style syntax but I can't imagine anyone describing this as beautiful art.
mlochbaum 34 days ago [-]
Well, do you know how it works? Don't judge a book by its cover and all. Although none of these are entirely aiming for elegance. The first is code golf and the other two have some performance hacks that I doubt are even good any more, but replacing ∧≢⥊ with ∧⌜ in the last gets you something decent (personally I'm more in the "utilitarian code is never art" camp, but I'd have no reason to direct that at any specific language).

The double-struck characters have disappeared from the second and third lines creating a fun puzzle. Original post https://www.ashermancinelli.com/csblog/2022-5-2-BQN-reflecti... has the answers.

badlibrarian 34 days ago [-]
When the junior programmers start saying "Turing complete" or the academics build a DSL in Julia that uses RegEx to parse Logic Symbols and stuffs the result in variables that use ancient characters that don't appear on your keyboard, it's a sure sign of imminent progress. Bonus if the PhD with nine years of schooling and five months of PHP experience at Facebook starts using emoji in commit messages.
hinkley 34 days ago [-]
“irony! Oh, no, no, we don't get that here. See, uh, people ski topless here while smoking dope, so irony's not really a, a high priority. We haven't had any irony here since about, uh, '83, when I was the only practitioner of it. And I stopped because I was tired of being stared at.”
skydhash 34 days ago [-]
Array Programming is an acquired taste, but once you do, solutions can be extremely simple, both to write and to explain.

Think about using matrix to describe geometric transformations instead of using standard functions.

badlibrarian 34 days ago [-]
You mean abstracted stuff like this instead of two lines of code that actually uses x and y?

  [ cos(θ)  -sin(θ)  0  0  ]
  [ sin(θ)   cos(θ)  0  0  ]
  [ 0        0       1  0  ]
  [ 0        0       0  1  ]
ssivark 34 days ago [-]
Yes, absolutely. Once you've expressed data as appropriate tensors (not all languages make this super convenient, unfortunately), it makes the implementation very readable, and easy to ensure that it's bug-free. It lets you see what is going on. Much better than futzing around with tensor components!
ashleyn 34 days ago [-]
It's giving APL: https://en.wikipedia.org/wiki/APL_(programming_language)
icen 34 days ago [-]
It is BQN, a descendant language
pavlov 34 days ago [-]
Why is it BQN instead of BQM? Clearly the idea was to increment every letter from APL, but then they had to go one further on the third letter.
BoiledCabbage 34 days ago [-]
This actual answer according the the author realized after he already liked the name.

He created it intending to be +1 of APL. Accidentally came up with BQN instead of BQM. Sat with that for 1hr, really liked the name, then realized that it should be BQM which he hated, so he stuck with BQN.

That said, it's and incredibly designed language. I honestly have never read any language (especially not designed by a single person) with the level of creative thought as he put into BQN. Some really incredible insights and deep understanding. It's amazing reading his posts / documentation about it. The focus on ergonomics, brand new constructs and the consistency/coherence of how all of his decisions fit together is really impressive.

layer8 34 days ago [-]
So, you write bequations in it? ;)
cenamus 34 days ago [-]
I'm somewhat sure the author actually mentioned that that was the intention, "Big Question Notation" and basically "apl" + 1. But he realized that it didn't match up
taolson 34 days ago [-]
Supposedly it stands for "Big Questions Notation", but that could just be a backronym.
hinkley 34 days ago [-]
I’m hoping they pronounce it “beacon” but the off by one error jokes also just write themselves.
rscho 34 days ago [-]
No, it's 'bacon' :-)
ModernMech 34 days ago [-]
They were following a Fibonacci sequence.
mlochbaum 34 days ago [-]
It's just. So gross. Say it. Sudden interruption of slime coming up your throat. Like walking out the door into a spiderweb. Alphabetically I was mistaken but in every way that matters I was right.
pavlov 34 days ago [-]
Hmm. I guess it if was BQM, it would be pronounced “bequem” which means comfortable in German.

And a comfortable APL is clearly an oxymoron.

mlochbaum 34 days ago [-]
Ordinarily I'd make fun of the Germans for giving such an ugly name to a nice concept, but I've always found "comfortable" to be rather unpleasant too (the root "comfort" is fine).
rscho 34 days ago [-]
This type of syntax allows rapid iteration when looking at different implementations and experimenting with array problems. It should be thought of more as math notation than general programming.
hyperbrainer 34 days ago [-]
I see it as beautiful the same way Galadriel would be beautiful as the Dark Queen. Utterly captivating and powerful, and yet something that should never be.
KineticLensman 34 days ago [-]
"All shall love me and despair"
teleforce 34 days ago [-]
Dlang does not has rank polymorphism and it handle array just fine with crazy speed in both compilation and execution.

It can be faster than Fortran based library that is still being used by Matlab, Rust and Julia [1].

It will be interesting to compare Mojo moblas BLAS library with GLAS library performance in D.

[1] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen (2016):

http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...

cdavid 34 days ago [-]
If I understand correctly what is meant by rank polymorphism, it is not just about speed, but about ergonomics.

Taking examples I am familiar w/, it is key that you can add a scalar 1 to a rank 2 array in numpy/matllab without having to explicitly create a rank 2 array of 1s, and numpy somehow generalizes that (broadcasting). I understand other array programming languages have more advanced/generic versions of broadcasting, but I am not super familiar w/ them

pjmlp 33 days ago [-]
Unfortunely, while the community is great, what it doesn't have is a direction, thus keeps pivoting every couple of years, and with that lost the adoption opportunity window it had a decade ago.
teleforce 33 days ago [-]
Form is temporary, class is permanent.