1, recognize the big data
Big data development engineer, first you have to be familiar with relational databases, such as Oracle or MySQL, familiar with the development of data warehouses after the benefit of; once again familiar with the Hadoop, this is now the field of big data in the use of one of the most technology, it HDFS Distributed storage can be realized, Yarn is an excellent resource scheduling framework
2, big data skills required
Must master the skills:
Java advanced (virtual machine, concurrency), Linux basic operations, Hadoop (HDFS + MapReduce + Yarn ), HBase (HDFS + MapReduce + Yarn ), HBase (HDFS + MapReduce + Yarn ), HBase (HDFS + MapReduce + Yarn ), HBase (HDFS + MapReduce + Yarn). Yarn ), HBase (JavaAPI operation + Phoenix ), Hive (Hql basic operation and understanding of the principles), Kafka, Storm/JStorm, Scala, Python, Spark (Core + sparksql + Spark streaming ), auxiliary gadgets (Sqoop/ Flume/Oozie/Hue, etc.)