- As a Hadoop Administration responsibility include installation, configuration, upgrade, backup and recovery, commissioning and decommissioning data nodes, cluster setup, cluster performance, monitoring and maintaining cluster healthy.
- Strong experience in configuring Hadoop ecosystem tools with including Pig, Hive, Hbase, Sqoop, Flume, Kafka, Oozie, Zookeeper, Atlas, Nifi.
- Strong understanding on HDF
- Expert in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake and Huge Data warehouses
- Strong understanding on data governance tool like Atlas
- Strong understanding on Hadoop architecture and MapReduce framework.
- Optimized the configurations of Map Reduce, Pig and Hive jobs for better performance.
- Experience in transferring data between HDFS and Relational Database with Sqoop.
- Experience in configuring Hadoop based monitoring tools Kibana, Grafana.
- Involved in cluster maintenance, bug fixing and troubleshooting
- Experience on commissioning, decommissioning, balancing and managing nodes and tuning server for optimal performance of the cluster.
- Strong knowledge on setting up automatic failover control and manual failover control using Zookeeper and quorum journal nodes.
- Implement and manage Secure Authentication and Authorization mechanism for Hadoop clusters using Kerberos and Rangers.
- Good Experience in setting up the Linux environments, Password less SSH, Creating file systems, disabling firewalls, swappiness, SELinux.
- Scheduling all Hadoop/hive/sqoop/Hbase jobs using Oozie.
- Experience monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network (DNS, TCP/IP).
- Experience in setup, configuration and management of security for Hadoop clusters using Kerberos and integration with LDAP/AD at an Enterprise level
- Involved in bench marking Hadoop/HBase cluster file systems various batch jobs and workloads.
- Good knowledge in understanding Mysql, Oracle. PostgreSQL
- Good experience on Customer service experience / strong customer focus.