• v7.0.8
    • v7.11.0 (latest)
    • v7.10.0
    • v7.9.0
    • v7.8.0
    • v7.7.0
    • v7.6.0
    • v7.5.0
    • v7.3.0
    • v7.2.0
    • v7.1.1
    • v7.1.0
    • v7.0.8
  • Services & Support
  • Devo.com
  • Contact
    • Contact Us
    • Request a Demo
    • Partner Inquiry
  • Log In
    • USA Devo
    • EU Devo
PREVIOUS
Forward response to email
NEXT
Forward response to Kafka

API reference / Query API / Running queries with the Query API / Forward response to HDFS

Download as PDF

Forward response to HDFS

Before sending data to HDFS, contact Devo customer support to validate the destination.

Using the destination object in a query request, you can have query responses forwarded to your organization's data storage services, including Hadoop Distributed File System (HDFS)-type systems. 

To send your response to HDFS, include the destination object in your request and set type to hdfs like this:

"destination": {
 "type":"hdfs"
}

Parameters

With the destination.type set to hdfs, you need to specify some additional parameters specific to HDFS to identify where and how to save the data contained in the response:

ParameterDescription

fs.defaultFS

Specifies where the HDFS server is listening (for example, hdfs://172.16.1.17:8020).

full path

Specifies the path where the data will be written (for example, /tmp/devo/test_05).

file

Specifies the file to which the data will be saved. Data files have the following format: {file_name}_{node_name}.data

dfs.replication

Defines the default block replication factor. The actual number of replications can be specified when the file is created.

Query using QueryID and forwarding to HDFS

{
    "queryId": "Query_Id_from_web_app",
    "from": 1481471880,
    "to": 1481558280,
    "destination": {
        "type":"hdfs",
        "params": {
            "fs.defaultFS":"hdfs://server:port",
            "dfs.replication":"1",
            "file":"/tmp/devo/test_05"
            ...........
        }
    }
    "mode": {
       "type":"json"
    }   
}

Query using LINQ and forwarding to HDFS

{
    "query": "LinQ_Query",
    "from": 1481471880,
    "to": 1481558280,
    "destination": {
        "type":"hdfs",
        "params": {
            "fs.defaultFS":"hdfs://server:port",
            "dfs.replication":"1",
            "file":"/tmp/devo/test_05"
            ...........
        }
    } 
    "mode": {
       "type":"json"
    }
}
Learn more about HDFS by reviewing the Hadoop HDFS Architecture Guide.
Download as PDF

PREVIOUS
Forward response to email
NEXT
Forward response to Kafka

Export

See what Devo can do for you. Request a demo!
Discover what's new (Release notes)
  • v7.0.8
    • v7.11.0 (latest)
    • v7.10.0
    • v7.9.0
    • v7.8.0
    • v7.7.0
    • v7.6.0
    • v7.5.0
    • v7.3.0
    • v7.2.0
    • v7.1.1
    • v7.1.0
    • v7.0.8
  • Services & Support
  • Devo.com
  • Contact
    • Contact Us
    • Request a Demo
    • Partner Inquiry
  • Log In
    • USA Devo
    • EU Devo
  • +1 888 6830910 (USA)
  • +34 900 838 880 (Spain)
Copyright © 2019 Legal Terms Privacy Policy Cookies Policy

Powered by Confluence and Scroll Viewport