LGTM, but adding adding a dependency is a significant change. @SethTisue wdyt?

This PR doesn't include the classfiles of directories into scala-compiler.jar, but it adds a dependency to the compiler's pom.

For scala-asm and java-diff-utils we include the classfiles into scala-compiler.jar (https://github.com/scala/scala/blob/v2.13.12/build.sbt#L503-L504). For asm we also remove the dependency from the pom, I guess it's an oversight that we don't do so for java-diff-utils (https://github.com/scala/scala/blob/v2.13.12/build.sbt#L566)?

java-diff-utils was included in scala-compiler.jar (and also added as a dependency in the pom) not so long ago, only in 2.13.9 (#9987). It's good to see that nothing broke 🙂.

For jna and jline we keep them as ordinary dependencies in the pom. Maybe that has something to do with sbt and how it provides jline to the console, I don't know...

I didn't think of the sbt angle. I was following jline just because. It's only a couple of classes, so ingestion is feasible. In summary, the feature (improved file location) is nice to have at zero cost; and also the PR is a probe of whether or how to incorporate a relatively trivial dependency. I resisted giving much thought to either issue.

I'll think on this next week.

My initial gut reaction is that it's okay because it's only for the REPL, and anyway it's a Java-based dependency.

If we take it, we need to make sure and put the license information (MPL) in the usual places.

One thought I have is that it ought to happen in Scala 3 first. Because if it's not going to also happen (in a reasonably short time) in Scala 3, then it isn't worth disrupting the Scala 2 status quo? But if does happen in Scala 3, we can argue that it's worth aligning with.

Unfortunately, the previous theory or policy, that Scala 3 would innovate and then we can backport, doesn't work because Scala 3 has its own priorities and resource limitations. It makes more sense to try out modest innovations on Scala 2 and then forward port what is useful (in the form of a fully baked design). I don't know that -Wconf turned out that way because of the difference in the feature set (but maybe it's the exception proving the rule), but -Wnonunit-statement is a clear example of a feature that benefited from experimentation and also proved useful enough (to someone) to port.

The innovation that would affect Scala 3 directly would be: share the history file! Use a custom history that knows the source version for each line. That might even be helpful for 3.x versions. Now that you mention it, why isn't repl history in tasty format? The use case is that I want to try something out in Scala 2, then switch to 3 and retry it. Or rather, conversely! It was also be cool if there were a rewrite or quickfix so when Scala 2 runs a line of Scala 3 syntax, I don't have to edit anything! You see what happens when you enable innovation.

I agree we don't need to block everything on Scala 3 parity, we can decide case by case. Here it's fine if we go ahead.

Seth also mentioned that scala-cli would eventually become the REPL launcher, does that doesn't affect the history file?

My initial gut reaction is that it's okay because it's only for the REPL

Note that there's no scala-repl.jar, it's all merged into scala-compiler.jar. We have two options

• a new dependency in the compiler pom (current state of this PR), or
• merge the classfiles into scala-compiler.jar

Not sure what assumption there are these days in build tools about the jar files of a Scala version. Maybe including the classes into the compiler jar is safer. On the other hand, without shading, if a project has a dependency on scala-compiler they don't have a way to pick a different version of directories or java-diff-utils.

I agree we don't need to block everything on Scala 3 parity, we can decide case by case. Here it's fine if we go ahead.

Okay.

we need to make sure and put the license information (MPL) in the usual places

I've added a commit that does this. I've also updated the PR description to mention it.

The innovation that would affect Scala 3 directly would be: share the history file!

(I think it's fine if this never happens.)

Seth also mentioned that scala-cli would eventually become the REPL launcher, does that doesn't affect the history file?

As far as I can see, it doesn't matter how the REPL gets launched, I don't see any potential for interaction there.

Maybe including the classes into the compiler jar is safer

In some sense yes. But at the cost of extra build hassle for us, and also at the cost of evading the normal mechanism for dependencies. I don't think shading is a clear win in this case. (And note that we've even discussed going in the other direction with e.g. ASM, which we shade because we have un-upstreamed changes. I think there's a scala-dev ticket about addressing that.)

Not sure what assumption there are these days in build tools about the jar files of a Scala version

Well, I don't remember any trouble around this when we've changed or updated the REPL dependencies in the past. (It's possible some tooling needed to adapt downstream, but if so it happened without ).

I've attached the release-notes label partly with the intent of alerting tooling maintainers. (It also alerts other folks for whom a new dependency may create work, e.g. shops that use internal artifact repositories to which they vet all additions, rather than allowing their developers direct access to Maven Central.)

@som-snytt (leaving the presence or absence of _jline3 aside for the moment) does the "The history file will now typically be found in..." section I added to the PR description look correct to you for Windows and Linux? I tested it the Mac one on my own laptop.

(Please review the whole PR description, actually!)

So we are adding directories as a pom dependency.

The new dependency listed in the POM as "optional"

@SethTisue I don't see that (there's something about "optional" in osgi). I don't think we ever used "optional" in our poms, so we shouldn't.

Let's also clean up the inconsistency around java-diff-utils (#10611 (comment)), i.e., stop merging its classfiles into scala-compiler.jar.

I don't think we ever used "optional" in our poms, so we shouldn't.

Oops, I was looking at the OSGi thing. I removed that bit from the PR description.

Let's also clean up the inconsistency around java-diff-utils (#10611 (comment)), i.e., stop merging its classfiles into scala-compiler.jar.

Okay — I'll submit a separate PR for that (after this one is merged).

IIRC I intended this as a baby step toward better file management, with an eventual goal of probing what it takes for the new dependency and also improving interop between versions with a custom history.

I haven't followed up with a custom history; that would be experimental. The use case is I want to try something with both 2&3 to see if it even works the same. "But how often do you do that?"

It was not a goal to create a time suck for Seth, so we could skip the feature for now. But if you're in for a penny, I'm in for a pound.

I do think it's a good change and worth pursuing.

(Though whether it makes 2.13.13 or has to wait for 2.13.14 doesn't concern me. I will provisionally milestone it for 2.13.14.)

TIL

  // Where we keep fsc's state (ports/redirection)
  lazy val scalacDir = (Path(Properties.userHome) / ".scalac").createDirectory(force = false)

[NEEDS POLISH] means he hopes VirtusLab steps in to help.