RATIS-372. Docker containers and docker-compose orchestration for the logservice
authorJosh Elser <elserj@apache.org>
Mon, 24 Dec 2018 04:46:00 +0000 (23:46 -0500)
committerJosh Elser <elserj@apache.org>
Thu, 3 Jan 2019 20:41:24 +0000 (15:41 -0500)
Signed-off-by: Rajeshbabu Chintaguntla <rajeshbabu@apache.org>
ratis-logservice/Dockerfile [new file with mode: 0644]
ratis-logservice/README.md
ratis-logservice/docker-compose.yml [new file with mode: 0644]
ratis-logservice/src/assembly/assembly.xml

diff --git a/ratis-logservice/Dockerfile b/ratis-logservice/Dockerfile
new file mode 100644 (file)
index 0000000..244c402
--- /dev/null
@@ -0,0 +1,19 @@
+FROM openjdk:8-jdk-alpine
+
+######
+RUN apk update && apk upgrade
+
+ENV BASEDIR /opt/ratis-logservice/
+
+# Our ratis-logservice tarball
+ARG BINARY
+ARG VERSION
+
+# Extract, symlink, and remove the ratis-logservice tarball
+COPY $BINARY $BASEDIR
+RUN tar xf ${BASEDIR}/ratis-logservice-${VERSION}-bin.tar.gz -C $BASEDIR \
+    && ln -s ${BASEDIR}/ratis-logservice-${VERSION} ${BASEDIR}/current \
+    && rm ${BASEDIR}/ratis-logservice-${VERSION}-bin.tar.gz
+
+# Use the nice symlink we made as our "starting directory"
+WORKDIR $BASEDIR/current
index 190f478..8dc9e9f 100644 (file)
@@ -1,18 +1,22 @@
 # Ratis LogService
 
+The Ratis LogService is an distributed, log implementation built on top of Apache Ratis. The LogService provides the
+ability to create named, durable, append-only data structures with the ability to perform linear reads.
 
-## Example shell
+## Launching the LogService
 
-Build:
-```bash
-$ mvn install
-```
+The LogService is compose of two Ratis quorums: the metadata quorum and the log quorum. These can be launched manually
+or via docker-compose.
 
-Change to logservice dectory:
+First, the project must be built:
 ```bash
-$ cd ratis-log-service
+$ mvn clean package -DskipTests
 ```
 
+Then, the service can be launched.
+
+### Manual
+
 Launch Metadata daemons:
 ```bash
 $ mvn exec:java -Dexec.mainClass=org.apache.ratis.logservice.server.MetadataServer -Dexec.args="-p 9991 -d $HOME/logservice1 -h localhost -q localhost:9991,localhost:9992,localhost:9993"
@@ -38,7 +42,7 @@ $ mvn exec:java -Dexec.mainClass=org.apache.ratis.logservice.server.LogServer -D
 $ mvn exec:java -Dexec.mainClass=org.apache.ratis.logservice.server.LogServer -Dexec.args="-p 9953 -d $HOME/worker3 -h localhost -q localhost:9991,localhost:9992,localhost:9993"
 ```
 
-Now, we have started three daemons which can service a single LogStream. They will rep to the Metadata q,
+Now, we have started three daemons which can service a single LogStream. They will register to the Metadata quorum,
 and the Metadata quorum will choose three of them to form a RAFT quorum to "host" a single Log.
 
 Note: the `q` option here references to the Metadata quorum, not the worker quorum as is the case for the Metadata daemons.
@@ -59,7 +63,7 @@ $ docker-compose up
 
 Then, a client container can be launched to connect to the cluster:
 ```bash
-$ mvn exec:java -Dexec.mainClass=org.apache.ratis.logservice.shell.LogServiceShell -Dexec.args="-q localhost:9990,localhost:9991,localhost:9992"
+$ docker run --rm --network ratis-logservice_default -it ratis-logservice java -cp "/opt/ratis-logservice/current/conf:/opt/ratis-logservice/current/lib/*" org.apache.ratis.logservice.shell.LogServiceShell -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999
 ```
 
-This command will launch an interactive shell that you can use to interact with the system.
+Take care that the correct network is provided to the LogServiceShell command.
diff --git a/ratis-logservice/docker-compose.yml b/ratis-logservice/docker-compose.yml
new file mode 100644 (file)
index 0000000..b82873b
--- /dev/null
@@ -0,0 +1,59 @@
+version: '3'
+
+# We already have a default network using the bridge adapter, so we don't need to redefine it.
+
+services:
+  master_1:
+    image: ratis-logservice:latest
+    command: "java -cp 'conf:lib/*' org.apache.ratis.logservice.server.MetadataServer -h master1.logservice.ratis.org -p 9999 -d /tmp/logservice-metadata -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999"
+    ports:
+     - 9999
+    networks:
+      default:
+        aliases:
+          - master1.logservice.ratis.org
+  master_2:
+    image: ratis-logservice:latest
+    command: "java -cp 'conf:lib/*' org.apache.ratis.logservice.server.MetadataServer -h master2.logservice.ratis.org -p 9999 -d /tmp/logservice-metadata -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999"
+    ports:
+     - 9999
+    networks:
+      default:
+        aliases:
+          - master2.logservice.ratis.org
+  master_3:
+    image: ratis-logservice:latest
+    command: "java -cp 'conf:lib/*' org.apache.ratis.logservice.server.MetadataServer -h master3.logservice.ratis.org -p 9999 -d /tmp/logservice-metadata -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999"
+    ports:
+     - 9999
+    networks:
+      default:
+        aliases:
+          - master3.logservice.ratis.org
+  worker_1:
+    image: ratis-logservice:latest
+    command: "java -cp 'conf:lib/*' org.apache.ratis.logservice.server.LogServer -p 9999 -h worker1.logservice.ratis.org -d /tmp/logservice-worker -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999"
+    ports:
+      - 9999
+    networks:
+      default:
+        aliases:
+          - worker1.logservice.ratis.org
+  worker_2:
+    image: ratis-logservice:latest
+    command: "java -cp 'conf:lib/*' org.apache.ratis.logservice.server.LogServer -p 9999 -h worker2.logservice.ratis.org -d /tmp/logservice-worker -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999"
+    ports:
+      - 9999
+    networks:
+      default:
+        aliases:
+          - worker2.logservice.ratis.org
+  worker_3:
+    image: ratis-logservice:latest
+    command: "java -cp 'conf:lib/*' org.apache.ratis.logservice.server.LogServer -p 9999 -h worker3.logservice.ratis.org -d /tmp/logservice-worker -q master1.logservice.ratis.org:9999,master2.logservice.ratis.org:9999,master3.logservice.ratis.org:9999"
+    ports:
+      - 9999
+    networks:
+      default:
+        aliases:
+          - worker3.logservice.ratis.org
index 77d1f94..c524c96 100644 (file)
       </includes>
     </dependencySet>
   </dependencySets>
+  <fileSets>
+    <fileSet>
+      <includes>
+        <include>README.md</include>
+      </includes>
+    </fileSet>
+    <fileSet>
+      <!-- Source directory, keeps 'src/test/resources' out of 'conf/' -->
+      <directory>src/test/resources</directory>
+      <!-- Destination directory -->
+      <outputDirectory>conf</outputDirectory>
+      <includes>
+        <include>log4j.properties</include>
+      </includes>
+    </fileSet>
+  </fileSets>
 </assembly>