Oozie Interview Questions

1) Types of Oozie Jobs
- Periodical/Coordinator Job: These are recurrent jobs which run based on a particular time or they can be configured to run when data is available.
- Coordinator jobs can manage multiple workflow based jobs as well as where the output of one workflow can be the input for another workflow.
- The chained behavior is known as “data application pipeline”.
- Oozie Hadoop Workflow: It is Directed Acyclic Graph (DAG) which consists of collection of actions.
- The Control nodes decide the chronological order, setting of rules, execution path decision, joining the nodes and fork.
- Whereas, Action node triggers the execution.
- Oozie Bundle: An Oozie bundle is collection of many coordinator jobs which can be started, suspended and stopped periodically.
- The jobs in this bundle are usually dependent on each other.

2) Oozie Architecture
- Oozie Architecture has a Web Server and a database for storing all the jobs.
- The default web server is Apache Tomcat, which is the open source implementation of Java Servlet Technology.
- Oozie server is a stateless web application and does not store any information regarding the user and job in-memory.
- All this information is stored in the SQL database and Oozie retrieves the job state from the database at the time of processing the request.
- The users or Oozie clients can interact with the server, using either the command line tool, Java Client API or, HTTP REST API.

Comments

Post a Comment

Popular posts from this blog

Hive Related Errors and fixes

1) FAILED: SemanticException [Error 10096]: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict Fix: a) set hive.exec.dynamic.partition.mode=nonstrict; b) set hive.exec.dynamic.partition=true; 2) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. Fix: deploy latest mysql-jdbs jar 3) 20)for ClassNotFoundException: org.apache.hadoop.hive.jdbc.HiveDriver add hive-jdbc JAR 4) java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException add hive-metastore jar 5) java.lang.NoClassDefFoundError: org/apache/thrift/TBase add libthrift JAR 6) java.lang.NoClassDefFoundError: org/apache/hadoop/hive/service/HiveInterface add hive-service Jar 7) java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface add libfb Jar 8) java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
Look into it »

Map Reduce Interview Questions

***************************************************Map Reduce******************************************************************** 1) Explain the usage of Context Object. - Context Object is used to help the mapper interact with other Hadoop systems. - Context Object can be used for updating counters, to report the progress and to provide any application level status updates. - ContextObject has the configuration details for the job and also interfaces, that helps it to generating the output. - it is introduced in Hadoop New Api - in old APi we use the OutputCollector, and the Reporter object use for communicate with Map reduce System. 2) What are the core methods of a Reducer? 1) run: - receives a Context containing the job's configuration as well as interfacing methods that return data from the reducer itself back to the framework. - This method is responsible for calling setup() once, reduce() once for each key associated with the reduce task, and cleanup()
Look into it »

Sqoop Interview Questions

1) Compare Sqoop and Flume Sqoop Flume Used for importing data from structured data sources like RDBMS. Used for moving bulk streaming data into HDFS. It has a connector based architecture. It has a agent based architecture. Data import in sqoop is not evetn driven. Data load in flume is event driven HDFS is the destination for importing data. Data flows into HDFS through one or more channels. 2) What is the default file format to import data using Apache Sqoop? 1) Delimited Text File Format - This is the default file format to import data using Sqoop. - This file format can be explicitly specified using the –as-textfile argument to the import command in Sqoop. - Passing this as an argument to the command will produce the string based representation of all the records to  the output files with the delimited characters between rows and columns. 2) Sequence File Format - It is a binary file format where records are stored in cust
Look into it »