Hadoop Tutorial
Time (sec)
Exit Code
Command
StdOutput
StdError
Clone Landoop Intro Repo
1.61
0
git clone https://gitlab.com/landoop/landoop-intro.git
Cloning into 'landoop-intro'...
hdfs put
3.11
0
hadoop fs -put ../README.md input.md
wordcount example
27.17
0
hadoop jar hadoop-examples.jar wordcount input.md results
build code
1.41
0
hadoop com.sun.tools.javac.Main WordCount.java
create jar
0.12
0
jar cf wc.jar WordCount.class WordCount$IntSumReducer.class WordCount$TokenizerMapper.class
run jar
25.52
0
hadoop jar wc.jar WordCount input.md results-2
Passed 6 out of 6
58.94 seconds
Sqoop Tutorial
Time (sec)
Exit Code
Command
StdOutput
StdError
import
22.18
0
sqoop-import --connect jdbc:mysql://mariadb.landoop.com/northwind --username data --password base --table Orders --m 1
hive create db
7.20
0
hive -e "CREATE DATABASE coyote_tutorial"
import into hive
12.50
0
sqoop import -jt local --connect jdbc:mysql://mariadb.landoop.com/northwind --username data --password base --query 'SELECT * FROM Orders WHERE $CONDITIONS' --m 1 --target-dir temp-orders --create-hive-table --hive-table "coyote_tutorial.orders" --hive-import --hive-overwrite --create-hive-table
drop hive database
6.95
0
hive -e "USE coyote_tutorial; DROP TABLE orders; DROP DATABASE coyote_tutorial"
Passed 4 out of 4
48.83 seconds
Kafka Tests
Time (sec)
Exit Code
Command
StdOutput
StdError
Cloudera Env Vars (no fail)
0.00
0
bash -c ' echo -e "\ Zookeeper (ZK) $ZK\n\ Broker List (BL) $BL\n\ Bootstrap Server (BS) $BS\n\ JAAS Login Configuration File (JAAS) $JAAS\n\ producer.properties path (PROD_PR) $PROD_PR\n\ consumer.properties path (CONS_PR) $CONS_PR\n\ KAFKA_OPTS (KAFKA_OPTS) $KAFKA_OPTS"'
Cloudera Perf Test (Kerberos)
0.00
fork/exec /opt/confluent-2.0.1/bin/kafka-producer-perf-test: no such file or directory
/opt/confluent-2.0.1/bin/kafka-producer-perf-test --topic test-rep-one --throughput 100000 --record-size 1000 --num-records 1000000 --producer-props bootstrap.servers="cloudera.landoop.com:9092"
Confluent Perf Test (Non-Kerberos)
0.00
fork/exec /opt/confluent-2.0.1/bin/kafka-producer-perf-test: no such file or directory
/opt/confluent-2.0.1/bin/kafka-producer-perf-test --topic test-rep-one --throughput 100000 --record-size 1000 --num-records 1000000 --producer-props bootstrap.servers="cloudera.landoop.com:29092"
Passed 1 out of 3
0.00 seconds
Spark Tests
Time (sec)
Exit Code
Command
StdOutput
StdError
Create Test Scala App
0.00
0
tee coyote-spark-test-app/src/main/scala/CoyoteTestApp.scala
Create Sbt File
0.00
0
tee coyote-spark-test-app/main.sbt
Put into HDFS a Test File
3.02
0
hadoop fs -put -f coyote-spark-test-app/src/main/scala/CoyoteTestApp.scala
Build SBT Package
13.13
0
sbt package
Spark-submit Application (Local)
7.17
0
spark-submit --class "CoyoteTestApp" --master local[4] coyote-spark-test-app/target/scala-2.11/coyote-test-project_2.11-1.0.jar
Verify via HDFS
2.66
0
hadoop fs -cat spark-results.txt/part-00000
Linecount: 19
Spark-submit Application (CDH Cluster)
19.65
0
spark-submit --class "CoyoteTestApp" --master yarn --deploy-mode cluster coyote-spark-test-app/target/scala-2.11/coyote-test-project_2.11-1.0.jar
Verify via HDFS
3.10
0
hadoop fs -cat spark-results.txt/part-00000
Linecount: 19
Passed 8 out of 8
48.74 seconds