KDnuggets Home » News » 2016 » Apr » Opinions, Interviews, Reports » HPE Haven OnDemand: Powerful Data Connectors for the Cloud and Enterprise ( 14:n32 )

HPE Haven OnDemand: Powerful Data Connectors for the Cloud and Enterprise


HPE Haven OnDemand simplifies how you can interact with data, allowing it to be transformed into an asset anytime, anywhere. Find out how the Connector APIs can facilitate this interaction.



Introduction

 
Connector APIs

HPE Haven OnDemand is a cloud services platform which simplifies how you can interact with data, allowing it to be transformed into an asset anytime, anywhere. HPE Haven OnDemand provides a collection of machine learning application programming interfaces (APIs) for interacting with structured and unstructured data in a variety of ways.

The HPE Haven OnDemand native APIs are POST and GET method-based, and can easily be embedded in any programming language or invoked independently with cURL calls. To make development even more flexible, Haven OnDemand also provides a number of programming language- and environment- specific APIs, including Android, Go, Node.js, R, Ruby and more.

There are a variety of ways to make your data accessible to Haven OnDemand APIs. One such way are connectors:

Connectors allow you to retrieve information from external systems and update it through Haven OnDemand APIs. You can use it to create an index of the data in the external repository, automate workflows, for notifications, or to facilitate migration to other locations.

Of course, these would need to be set up and managed in order to be useful. HPE Haven OnDemand offers a series of APIs for the simple and straightforward management of connectors, including:

  • Create Connector API - creates a connector configuration to connect to the external repository
  • Start Connector API - starts a run of a cloud flavor connector according to the configuration that you created with the Create Connector API
  • Update Connector API - updates the parameters for an existing connector
  • Stop Connector API - stops a connector from running that was started using the Start Connector API
  • Delete Connector - deletes an existing connector configuration

And which external systems are available to these connectors? HPE Haven OnDemand currently supports the following connector flavors:

  • Web Cloud Connector - retrieves content from a public Web address and indexes it into Haven OnDemand; follows links from a start page that you provide, and crawls to find and retrieve other Web pages on the same host
  • File System Onsite Connector - retrieves documents from a computer file system and indexes them into Haven OnDemand; provide the starting directory, and the connector collects documents from that directory and, optionally, any subdirectories
  • SharePoint Onsite Connector - retrieves documents from a Microsoft SharePoint server or a SharePoint Online account, and indexes them into Haven OnDemand
  • Dropbox Cloud Connector - retrieves content from a Dropbox application and indexes it into Haven OnDemand

A key point here is that connectors do not simply provide a pathway from the data to the APIs; connectors take care of the indexing of data being retrieved in order to facilitate eventual data processing.

This post will explore the HPE Haven OnDemand Connector APIs.

Additionally, it should also be noted that all HPE IDOL connectors include support for and can be used with Haven OnDemand, although setup requires a few additional steps. This expands coverage significantly to many Enterprise platforms and provides access to over 400 data repositories.

Using The Connector APIs

 
Now that we have an understanding of what the Connector APIs are for and what they can access, let's have a look at how they are used.

Start connector

Create Connector API
 
The Create Connector API is used to create a connector configuration for connecting to an external repository. Connectors can be scheduled to run, or in the case of cloud connector flavors can be run manually. The Create Connector API has the required parameters of config, destination, action, and index.

Before creating a connector, however, a text index must be created via the Create Text Index API, which can be accomplished via the following cURL call:

curl -X POST --form "index=myindex" --form "flavor=explorer" 
  --form "apikey=YOUR_API_KEY" https://api.havenondemand.com/1/api/sync/createtextindex/v1


Note that YOUR_API_KEY is your specific HPE Haven OnDemand API key. Learn more about API keys by reading Getting Started with Haven OnDemand.

A sample response from the above POST method cURL call is shown below:

{
  "message": "index created",
  "index": "myindex"
}


Now we can use the created index, named myindex, in the Create Connector API config.

curl 
  -X POST --form "flavor=web_cloud" --form "connector=myconnector" 
  --form "config={"url":"http://www.cnn.com"}" 
  --form "destination={"action":"addtotextindex","index":"myindex"}" 
  --form "description=Test" --form "apikey=YOUR_API_KEY" 
  https://api.havenondemand.com/1/api/sync/createconnector/v1


The response of the above POST method cURL call is shown below:

{
  "message": "connector created",
  "connector": "myconnector"
}


Success! Now we have a web cloud connector named myconnector being indexed on the index named myindex.

Connectors can be set to run on a schedule. Note that if the schedule parameter is not set initially, that cloud flavor connectors (web cloud, dropbox cloud) require a start connector to initialize, while onsite connectors (file system, sharepoint) run each time you start the onsite connector service.

There are a wide variety of available parameters and options beyond the required; see the documentation for each individual connector flavor, listed and linked in the Introduction section.


Sign Up