Oozie Interview Questions

1) Types of Oozie Jobs
- Periodical/Coordinator Job: These are recurrent jobs which run based on a particular time or they can be configured to run when data is available.
- Coordinator jobs can manage multiple workflow based jobs as well as where the output of one workflow can be the input for another workflow.
- The chained behavior is known as “data application pipeline”.
- Oozie Hadoop Workflow: It is Directed Acyclic Graph (DAG) which consists of collection of actions.
- The Control nodes decide the chronological order, setting of rules, execution path decision, joining the nodes and fork.
- Whereas, Action node triggers the execution.
- Oozie Bundle: An Oozie bundle is collection of many coordinator jobs which can be started, suspended and stopped periodically.
- The jobs in this bundle are usually dependent on each other.

2) Oozie Architecture
- Oozie Architecture has a Web Server and a database for storing all the jobs.
- The default web server is Apache Tomcat, which is the open source implementation of Java Servlet Technology.
- Oozie server is a stateless web application and does not store any information regarding the user and job in-memory.
- All this information is stored in the SQL database and Oozie retrieves the job state from the database at the time of processing the request.
- The users or Oozie clients can interact with the server, using either the command line tool, Java Client API or, HTTP REST API.

Comments

Post a Comment

Popular posts from this blog

Hive Related Errors and fixes

1) FAILED: SemanticException [Error 10096]: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict Fix: a) set hive.exec.dynamic.partition.mode=nonstrict; b) set hive.exec.dynamic.partition=true; 2) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. Fix: deploy latest mysql-jdbs jar 3) 20)for ClassNotFoundException: org.apache.hadoop.hive.jdbc.HiveDriver add hive-jdbc JAR 4) java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException add hive-metastore jar 5) java.lang.NoClassDefFoundError: org/apache/thrift/TBase add libthrift JAR 6) java.lang.NoClassDefFoundError: org/apache/hadoop/hive/service/HiveInterface add hive-service Jar 7) java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface add libfb Jar 8) java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory ...
Look into it »

HBase Interview Questions

1) When should you use HBase and what are the key components of HBase? - HBase should be used when the big data application has 1) A variable schema 2) When data is stored in the form of collections 3) If the application demands key based access to data while retrieving. - Key components of HBase are HRegion- This component contains memory data store and Hfile. default size 1024 mb HRegion Server-This monitors the Region. HBase Master-It is responsible for monitoring the region server. Zookeeper- It takes care of the coordination between the HBase Master component and the client. Catalog Tables-The two important catalog tables are ROOT and META. ROOT table tracks where the META table is and META table stores all the regions in the system. 2) What are the different operational commands in HBase at record level and table level? - Record Level Operational Commands in HBase are –put, get, increment, scan and delete. - Table Level Operational Comma...
Look into it »

Hive Interview Questions

1) Explain about the SMB Join in Hive. - SMB represents Sort Merge Bucket join in Hive. - In SMB join in Hive, each mapper reads a bucket from the first table and the corresponding bucket from the second table  and then a merge sort join is performed. - it s mainly used as there is no limit on file or partition or table join. - SMB join can best be used when the tables are large. - In SMB join the columns are bucketed and sorted using the join columns. - All tables should have the same number of buckets in SMB join. 2) How can you connect an application, if you run Hive as a server? - ODBC Driver-This supports the ODBC protocol - JDBC Driver- This supports the JDBC protocol - Thrift Client- This client can be used to make calls to all hive commands using different programming language like PHP, Python, Java, C++ and Ruby. 3) What does the overwrite keyword denote in Hive load statement? - Overwrite keyword in Hive load statement deletes the contents of t...
Look into it »