Thursday, August 23, 2018

How to avoid an extra trailing newline when using Scala's sys.process to shell out

Leave a Comment

I need to execute an external binary from my Scala code and get its output. The standard way to do this seems to be using scala.sys.process. The following code mostly works:

import scala.sys.process._ val command = Seq("python3", "-c", "print('foo', end='')") val result = command.!! 

However, it seems like there is an extra trailing newline:

print(result.length) // prints 4 instead of 3 print(result) // prints an extra newline at the end 

I can just modify the result string to remove the last character, but it seems weird that sys.process would add a trailing newline to output, so I thought it might either be a bug with what I'm doing or something that can be configured.

Is the trailing newline supposed to be there?

Is there a way to get rid of it without manipulating the output string?

3 Answers

Answers 1

Have you tried val result = cmd.lineStream? (Or the related .lineStream_!, which offers some protection against exceptions.)

The result is a Stream[String]. Each element of the stream is a string of output from the process with whitespace preserved but there are no newline characters because that's the stream element delimiter.


The added newline appears to be intentional for processes launched with !!.

In ProcessBuilderImpl.scala:

def !! = slurp(None, withIn = false) 

...

private[this] def slurp(log: Option[ProcessLogger], withIn: Boolean): String = {   val buffer = new StringBuffer   val code   = this ! BasicIO(withIn, buffer, log)    if (code == 0) buffer.toString   else scala.sys.error("Nonzero exit value: " + code) } 

In BasicIO.scala:

def apply(withIn: Boolean, buffer: StringBuffer, log: Option[ProcessLogger]) =     new ProcessIO(input(withIn), processFully(buffer), getErr(log)) 

...

def processFully(buffer: Appendable): InputStream => Unit =   processFully(appendLine(buffer)) 

...

private[this] def appendLine(buffer: Appendable): String => Unit = line => {     buffer append line     buffer append Newline  <--!! } 

Answers 2

sys.process isn't adding the trailing newline: echo is. From the documentation:

DESCRIPTION The echo utility writes any specified operands, separated by single blank (') characters and followed by a newline (\n') character, to the standard output.

Using trim can remove this. Otherwise, you can do this, if your shell supports it:

val result = "echo -n foo".!!

Answers 3

In order to preserve the output as presented, you have to intercept the InputStream before any NewLine interpretations (or additions).

import sys.process._  val cmnd = Seq("/bin/echo","-n","one\n\nthree")  val stdOutBuf = new StringBuilder val pio = new ProcessIO(_.close()  //ignore STDIN   ,out=>{                          //save STDOUT     val src = io.Source.fromInputStream(out)     src.foreach(stdOutBuf.+=)     src.close()   },_.close())                     //ignore STDERR (or not, up to you)  val exitCode :Int = cmnd.run(pio).exitValue() val exactOutput :String = stdOutBuf.result() 
If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment