Should all syntax have a Location attribute on some Node?

[nixme]

It's great that most tokens in the Ruby syntax are recorded in *_loc attributes, but some are missing. Is this expected and outside the scope of Prism?

For example, the following two produce identical trees

Prism.parse("foo[1, 2]=  3")
Prism.parse("foo[1, 2]  =3")

because the location of the = is not recorded anywhere.

Similarly for arrays, [1,2 , 3]— I don't see a way to get the locations of the commas.

[kddnewton]

In general we provide location information for anything that can be directly attached to nodes/have some kind of meaning for nodes. This means we don't provide location information for any kind of delimiters. This includes things like commas, newlines, and semicolons.

The = sign that you're referring to here is an edge case that I wish we had a better solution for. Because we want to reuse CallNode as much as possible, we don't have a spot for attaching the = sign. You'll notice the same thing on foo.bar = 1. Because it's just a method call, the method name includes the =, but it could also include a bunch of whitespace. As a compromise, we decided to drop the location of the =.

As a workaround, if you really need the location of the = for a project, you can scan forward in the source from the end of the message_loc. It will always be the first non-whitespace character.