When you develop a Scala application you mostly use SBT, Gradle or Maven or similar build tools to build, test and run you application locally. Whether is is a Play! framework, Spray, Akka, etc. application, I certainly mostly depend on SBT.
Stopping and starting in SBT is easy, simply execute: run (or if using Revolver: re-start ).
But in a production environment and staging environments in between you cannot do the same. You have to package up a binary that somehow gets deployed and run on the servers. They must also be able to be stopped and started easily and automated.
(I answered a Stack Overflow question recently that is the basis of this document).
This assumes your environment servers are Linux based. It also assumes it is based on Debian or its derivatives e.g. Ubuntu. Transplanting to any other Linux distribution and/or different service management should however be relatively easy.
This assumes you have one jar file with all libraries needed, so that it can be deployed as a stand alone application.
This is easily achieved by using the SBT assembly plugin for SBT. For a Play! application you achieve the same with the dist build command. Keep an eye especially if using Maven that if any of your dependencies' scope are not set as provided as they won't be included.
This solution consists of two parts.
One is to make a daemon class in the application that handles bootstrapping the application itself inside the JVM, and to gracefully shutdown inside the JVM as well.
The other is to create an init.d script so that it can be stopped and started as a service.
You application and especially its frameworks may already supply a similar type of class so you may not need this.
In my example the application is a Spray web service that sits on top of Akka. Spray does supply its own example Boot class to stop and start the application. However it is not sufficient enough for my needs.
This solution does create a number of objects, classes, abstract classes and traits. This is so it is clear what does what and follows the conventions of small class and methods that do one thing. And allows further reuse and extension with more complexities.
This daemon utilises the Apache Commons Daemon.
Add this dependency to your build.sbt, Build.scala, pom.xml or whatever is relevant:
"commons-daemon" % "commons-daemon" % "1.0.15"
Then we create a number of standard classes that can be shared between applications so it may be suitable as a library.
package com.example.myapplication.server
import org.apache.commons.daemon._
trait ApplicationLifecycle {
def start(): Unit
def stop(): Unit
}
abstract class AbstractApplicationDaemon extends Daemon {
def application: ApplicationLifecycle
def init(daemonContext: DaemonContext) {}
def start() = application.start()
def stop() = application.stop()
def destroy() = application.stop()
}
class ApplicationDaemon() extends AbstractApplicationDaemon {
def application = new Application
}
object ServiceApplication extends App {
val application = createApplication()
def createApplication() = new ApplicationDaemon
private[this] var cleanupAlreadyRun: Boolean = false
def cleanup(){
val previouslyRun = cleanupAlreadyRun
cleanupAlreadyRun = true
if (!previouslyRun) application.stop()
}
Runtime.getRuntime.addShutdownHook(new Thread(new Runnable {
def run() {
cleanup()
}
}))
application.start()
}
The actual composition and custom logic of the application is in the Application class below. If using Spray the Boot class can be removed or rather move the logic part of it into parts of the new daemon class.
package com.example.myapplication.server
import akka.actor.{Props, ActorSystem}
import spray.can.Http
import akka.io.IO
import com.example.myapplication.api.MyServiceActor
class Application() extends ApplicationLifecycle with Logging {
private[this] var started: Boolean = false
private val applicationName = "MyApplication"
implicit val actorSystem = ActorSystem(s"$applicationName-system")
def start() {
logger.info(s"Starting $applicationName Service")
if (!started) {
started = true
val myService = actorSystem.actorOf(Props[MyServiceActor], "my-service")
IO(Http) ! Http.Bind(myService, interface = "0.0.0.0", port = 8280)
}
}
def stop() {
logger.info(s"Stopping $applicationName Service")
if (started) {
started = false
actorSystem.shutdown()
}
}
}
The green section in the middle of the Application class above are the specifics for this application and will of course be different for each application. E.g. initialisers, orchestration etc
E.g. an Akka application would create its core orchestrating actors here. Spray does the same and as detailed above it also binds its http service to the desired port. If using cake pattern you could create the specific registry context here.
Now that your application is able to respond to daemon we need to set up the side that interacts with it outside of the JVM. In our case that is a standard Linux init.d script.
For this purpose we will use the tried and tested JSVC from the Apache commons-daemon.
Create a file as /etc/init.d/myapplication:
#!/bin/sh
### BEGIN INIT INFO
# Provides: myapplication
# Required-Start: $local_fs $remote_fs $network
# Required-Stop: $local_fs $remote_fs $network
# Should-Start: $named
# Should-Stop: $named
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Control myapplication
# Description: Control the myapplication daemon.
### END INIT INFO
set -e
if [ -z "${JAVA_HOME}" ]; then
JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:/bin/java::")
fi
JAVA_OPTS="-Xms512m -Xmx1024m"
APP=myapplication
PID=/var/run/${APP}.pid
OUT_LOG=/var/log/${APP}/${APP}_out.log
ERR_LOG=/var/log/${APP}/${APP}_err.log
DAEMON_USER=yourserviceuser
APP_LOG_CONFIG=/etc/mycompany/${APP}_logback.xml
APP_CONFIG=/etc/mycompany/${APP}.conf
APP_HOME=/opt/${APP}
APP_CLASSPATH=$APP_HOME/${APP}.jar
APP_CLASS=com.example.myapplication.server.ApplicationDaemon
if [ -n "$APP_LOG_CONFIG}" ]; then
JAVA_OPTS="-Dlogback.configurationFile=${APP_LOG_CONFIG} ${JAVA_OPTS}"
fi
DAEMON_ARGS="-home ${JAVA_HOME} -Dconfig.file=${APP_CONFIG} ${JAVA_OPTS} -pidfile ${PID}"
DAEMON_ARGS="$DAEMON_ARGS -user ${DAEMON_USER} -outfile ${OUT_LOG} -errfile ${ERR_LOG}"
DAEMON_ARGS="$DAEMON_ARGS -cp ${APP_CLASSPATH} ${APP_CLASS}"
. /lib/lsb/init-functions
case "$1" in
start)
log_daemon_msg "Starting ${APP}"
cd ${APP_HOME} && jsvc ${DAEMON_ARGS}
log_end_msg 0
;;
stop)
log_daemon_msg "Stopping ${APP}"
cd ${APP_HOME} && jsvc -stop ${DAEMON_ARGS}
log_end_msg 0
;;
*)
log_success_msg "Usage: {start|stop}"
echo "Usage: {start|stop}"
exit 1
;;
esac
exit 0
Note the DAEMON_ARGS could be on one line, it was split up to fit better on this page.
You will need to make sure you have a suitable service user instead of myserviceuser. Also the following folders need to exist:
If you do not use Logback or do not have an external configuration file you can remove APP_LOG_CONFIG and/or APP_CONFIG. Otherwise you should also have these files present:
First make the init script executable:
sudo chmod +x /etc/init.d/myapplication
You can then start the application with:
sudo service myapplication start
and stop the application with:
sudo service myapplication stop
For this application to automatically start on boot up in case of restarts etc, add the init script to the correct run level:
sudo update-rc.d myapplication defaults
Note this command changes dependant on your Linux distrobution.
You can monitor the logs and most likely initial typo errors at:
That is basically it. You can now start and stop a Scala application as a service on your production or staging environments.
Naturally you will extend this set up to suit your needs.
You probably want to add your init script to some sort of version control. I add mine to the same configuration repository that I also add the external configuration and logback configuration per environment.
Your servers might already be configured via Puppet, Chef, etc. So an obvious extension is to add the above configuration and scripts to the these configurations as well.
The daemon classes that are common between your applications can easily be extracted to a shared library.
I have made one for myself at github.com/flurdy/sander's lifecycle.scala.
There are other ways to run a Scala application on a remote server. There are different benefits and trade off with each though.
You will need to deploy a Skinny war/jar instead of full fat jar, or worse somewhere in between. This is because your app server may supply some of the libraries or even randomly override a few of your included libraries. You can achieve this with a heavy customisation of your Maven dependencies.
You can run sbt directly on the server. But this is very fragile and not what it intended for. It also breaks the one binary for all environments rule.
A quick and simple way is to run your application
directly with java from the command line.
By wrapping it with nohup the application
is not killed when your terminal session ends.
nohup java -jar mypath/myapp.jar &
Of course there is no restarts,
or start if server is rebooted.
Stopping it is only possible via something like kill -9 myprocesspid
You can extend the nohup command into a simple init.d script. This will allow you full control to stop and start the application manually and automatic. It is in fact a very simplified version of what I suggest below.
There is a SBT plugin that creates a init.d script for you that uses SBT's target/start commands.
There are plugins available to SBT that will automatically build a native package for you, such as .apt if Debian based. It does seem not quite robust enough to rely on yet however.
If you host your application via a PAAS, e.g. Heroku, this is usually all taken care for you. However if you use a IAAS, e.g. Amazon AWS ec2 you do have to do this manually.
You can follow my suggestion and tweak it slightly by using Java Service Wrapper instead of JSVC.
Describe how to make the init script compatible with systemd.
For typos and minor change suggestions, please clone this document repository at: github.com/flurdy/flurdy.com-docs, and send a pull request.
And/or alternatively please enhance the initial Stack Overflow answer.
For large changes perhaps create your own article and send me the link afterwards.
To hire me as a consultant / contractor contact me via www.eray.co.uk.
Or contact me via flurdy.com/contact.