Flume Related Errors

1) 401:Authentication credentials (https://dev.twitter.com/pages/auth) were missing or incorrect.
   Ensure that you have set valid consumer key/secret, access token/secret, and the system clock is in sync.
Fix: Check for the 3 mentioned details in your Conf file and system clock and try to relaunch again

2) 17/05/05 12:19:54 ERROR hdfs.HDFSEventSink: process failed
java.lang.VerifyError: class org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$RenewLeaseRequestProto overrides final method getUnknownFields.
()Lcom/google/protobuf/UnknownFieldSet;

Fix: 1) check for your HDFS sink path in the flume conf file and fixx it accordingly.
    2) check whether the the HDFS is in Safe mode and retry to leave from it.

3)  ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
Fix: Download and copy the flume-twitter-dource-1.6.0.jar into lib folder.
2) TwitterAgent.sinks.HDFS.hdfs.useLocalTimeStamp = true

4) java.lang.NoSuchMethodError: twitter4j.conf.Configuration.isStallWarningsEnabled()
Fix: 1) Download and copy the flume-twitter-dource-1.6.0.jar into lib folder.
2) TwitterAgent.sinks.HDFS.hdfs.useLocalTimeStamp = true

5) java.lang.NullPointerException: Expected timestamp in the Flume event headers, but it was null
Fix: TwitterAgent.sinks.HDFS.hdfs.useLocalTimeStamp = true

7) Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: twitter4j.json.JSONObjectType.determine(Ltwitter4j/internal/org/json/JSONObject;)Ltwitter4j/json/JSONObjectType;
Fix: Deploy latest Twiter-x*.jar and re run it

8)  404:The URI requested is invalid or the resource requested, such as a user, does not exist.
Unknown URL. See Twitter Streaming API documentation at http://dev.twitter.com/pages/streaming_api
Fix: Deploy latest Twiter-x*.jar and re run it

Comments

Post a Comment

Popular posts from this blog

Hive Related Errors and fixes

1) FAILED: SemanticException [Error 10096]: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict Fix: a) set hive.exec.dynamic.partition.mode=nonstrict; b) set hive.exec.dynamic.partition=true; 2) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. Fix: deploy latest mysql-jdbs jar 3) 20)for ClassNotFoundException: org.apache.hadoop.hive.jdbc.HiveDriver add hive-jdbc JAR 4) java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException add hive-metastore jar 5) java.lang.NoClassDefFoundError: org/apache/thrift/TBase add libthrift JAR 6) java.lang.NoClassDefFoundError: org/apache/hadoop/hive/service/HiveInterface add hive-service Jar 7) java.lang.NoClassDefFoundError: com/facebook/fb303/FacebookService$Iface add libfb Jar 8) java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory ...
Look into it »

HBase Interview Questions

1) When should you use HBase and what are the key components of HBase? - HBase should be used when the big data application has 1) A variable schema 2) When data is stored in the form of collections 3) If the application demands key based access to data while retrieving. - Key components of HBase are HRegion- This component contains memory data store and Hfile. default size 1024 mb HRegion Server-This monitors the Region. HBase Master-It is responsible for monitoring the region server. Zookeeper- It takes care of the coordination between the HBase Master component and the client. Catalog Tables-The two important catalog tables are ROOT and META. ROOT table tracks where the META table is and META table stores all the regions in the system. 2) What are the different operational commands in HBase at record level and table level? - Record Level Operational Commands in HBase are –put, get, increment, scan and delete. - Table Level Operational Comma...
Look into it »