Java Rest Client to get topology data

Hi,

Download the EMC ProSphere REST API 1.6 from from the product page https://support.emc.com/products/15995 It describe CAS authentication for ProSphere

Review the CAS Authentication section and the following is detailed which show you examples of how to write batch files or Perl scripts examples so the request will not be blocked for login username / passwords details.

You will not be able to directly pass the Username and password using the URL will not work as CAS authentication is required. Using the browsers add-ons will allow you to enter the login/password details during the page request by following steps 1 to 8 below.

Some URIs require you to issue POST, PUT, and DELETE HTTP requests. To do this from a web browser, you need plugins for Internet Explorer and Firefox. The recommended plugins are:

__MCE_ITEM____MCE_ITEM__·         httpAnalyzer for Internet Explorer browsers

__MCE_ITEM____MCE_ITEM__·         Poster for Firefox and Chrome browsers

• FireFox Addon: https://addons.mozilla.org/en-US/firefox/addon/restclient/ get HTTP requests directly from the ProSphere's REST API
• Chrome Addon: https://chrome.google.com/webstore/detail/fhjcajmcbmldlhcimfajhfbgofnpcjmb get HTTP requests directly from the ProSphere's REST API 

White paper called "ProSphere Integration: An overview of Rest APIs and Information Model" available on the product page which provides an overview of the ProSphere information architecture and REST APIs for ProSphere 1.0 but the ProSphere 1.6 REST API has been redesigned from the 1.0 version, providing a more robust API with increased functionality that allows programmers to extract ProSphere topology, performance, and alert data for integration into their storage environments.

CAS Authentication

The ProSphere 1.6 REST API is protected by the Central Authentication Service (CAS).

http://en.wikipedia.org/wiki/Central_Authentication_Service

All requests to the ProSphere REST interface must be authenticated, either by passing valid credentials in the request, or by passing in a cookie containing valid credentials.

The following basic steps describe CAS authentication in a ProSphere environment:

__MCE_ITEM__1.     Execute a GET against the target URL.

__MCE_ITEM__2.     Receive 401. Look at location: header.

__MCE_ITEM__3.     Do POST to that location. Include the username and password in body of that POST.

__MCE_ITEM__4.     Examine location header of the response. This is your session resource.

__MCE_ITEM__5.     Do a POST to the session resource location. The body of the POST must contain the URL you want to use.

__MCE_ITEM__6.     Take the contents of the response body, and add it as a query string argument to the URL you are trying to get.

__MCE_ITEM__7.     Execute the original GET request. Append the query string parameter ?ticket=. Include the cookies you are given.

__MCE_ITEM__8.     For additional requests, always resend the cookies received in step 7.

CAS Authentication Examples

The following batch file allows you to log in to ProSphere and submit a URL to examine. Note that the ProSphere server name is hard coded in this simple example.

@ECHO OFF

REM A script to do a RESTful CAS login

REM There is no error handling. 

REM

REM Usage:  curlcas.bat [username] [password] [URL-for-the-application-entry-point]

REM

REM For example:

REM 

REM curlcas.bat username mypassword https://ProSphereServerName/srm/topology/msa/types/StorageArray/instances 

REM

REM After that you can do any normal curl operations such as GETs, as long as you pass the one additional switch to read the cookies.txt file.  

REM 

REM curl -b cookies.txt -G https://ProSphereServerName/srm/topology/msa/types/StorageArray/instances

REM 

REM This will present the required CAS cookie to the application server, and curl just works as usual against a CAS protected site.

REM 

REM 

SETLOCAL

REM define the environment variable for where curl is installed.

SET MYCURLDIR=c:\curl

REM define the environment variable for the login command.  Note that the location of the CAS server is hard coded here.

REM This could/should be passed as an input environment variable, or perhaps as CLI argument %4.

SET LOGIN_CMD=%MYCURLDIR%\curl -s --insecure -i -POST https://ProSphereServerName/cas-server/v1/tickets --data username^^=%1 --data password^^=%2

REM

REM Do the login command and parse the response for the location header.

REM

FOR /F "usebackq eol= skip=3 tokens=1,2 delims= " %%i in (`%LOGIN_CMD%`) DO IF /I [%%i] == [Location:] SET PLACE_TO_POST=%%j

REM Create the command to get a Service Ticket from the RESTful interface for CAS.  Parse the return result for the ticket.

REM

SET GET_ST_CMD=%MYCURLDIR%\curl -s --insecure --data service^^=%3 -POST %PLACE_TO_POST%

FOR /F "usebackq tokens=1 delims=" %%i in (`%GET_ST_CMD%`) DO SET SERVICE_TICKET=%%i

REM

REM Present the ticket to the target service.  Notice you need to store the newly established session via the cookies file. 

SET FINAL_COMMAND=%MYCURLDIR%\curl -s --insecure -c cookies.txt -G %3?ticket=%SERVICE_TICKET%

%FINAL_COMMAND% > NUL

REM

REM All done.  Now you can run any curl command, passing in the same cookies file.

REM such as doing:

curl -k -b cookies.txt -G %3

REM

ENDLOCAL

Perl example

The following example is coded in Perl. This script automatically determines the location of the CAS server based on the URL passed in the original GET request.

#!/usr/bin/perl

#

# * Copyright 2009, EMC Corporation ("EMC").

# * All Rights Reserved

# *

# * UNPUBLISHED CONFIDENTIAL AND PROPRIETARY PROPERTY OF EMC. The copyright

# * notice above does not evidence any actual or intended publication of this

# * software. Disclosure and dissemination are pursuant to separate agreements.

# * Unauthorized use, distribution or dissemination are strictly prohibited.

#

# Purpose: reload.monitor will query the registry service and rewrite mon  

#          configuration files if the registry contents have changed

#

# reload.monitor -u 
    
     [
     ]
     
    
   

#

# -u 
    :  url of the registry service
   

# -e 
         :  location of the mon config file to write
   

# -d               :  enable debug

#

use URI;

use Getopt::Std;

use English;

use Data::Dumper;

use XML::Simple;

use File::Compare;

sub httpGET;

$opt_s = undef;

$opt_u = undef;

$opt_p = undef;

getopts ("s:u:p:");

$uname=$opt_u;

$pass=$opt_p;

if (!defined($uname)) {

    $uname = "appadmin";

}

if (!defined($pass)) {

    $pass = "Changeme1!";

}

my $cookieFile = "my_cookie.txt";

# this option has to be there

my $serviceLink = $opt_s;

Syntax() if (!defined($serviceLink));

$url = $serviceLink;

my ($result, $content) = checkURL($url, $cookieFile);

#  Functions

#####################################################################################

sub checkURL {

    my($url, $cookieFile) = @_;

    #Constants

    debugPrint("url: $url \n");

    debugPrint("cookieFile: $cookieFile\n");

    # assume to have a valid cookie to run first

    my($code, $location, @content) = ConnectToUrlWithCookie($url, $cookieFile);

    debugPrint("code from ConnectToURLWithCookie: $code");

    if ($code == 401) {

       # no cookie, or cookie expired

       #a)get CAS URI from 401 return code

       #b)get "internal" credentials from lockbox

       #c)get TGT, ST

       #d)try to connect the service URL again

       $cas_link = $location if (!defined($cas_link));

       # Now it is 401 (unauthorized access) 

       debugPrint("cas_link: $cas_link\n");

       if (!defined($uname) || !defined($pass)){

           return(1, "Cannot get username/password from lockbox \n");

       }

       debugPrint (" ============= GetCasTGT ============\n");

       my $tgt = GetCasTgt($cas_link, $uname, $pass);

       if (!defined($tgt) || $tgt =~ /^\s*$/) {

           return (1, "Cannot get Ticket Granting Ticket from CAS\n");

       }

       debugPrint ("TGT = $tgt \n");

       debugPrint (" ============= GetCasST ============\n");

       my $st = GetCasSt($url, $cas_link, $tgt);

       if (!defined($st) || ($st =~ /^\s$/)) {

           return (1, "Cannot get Service Ticket from CAS\n");

       }

       debugPrint ("ST => $st \n");

       ($code, @content) = ConnectToUrl($url, $st, $cookieFile);

       debugPrint ("Connect to URL code $code \n");

       ConnectToUrlWithCookie($url, $cookieFile);

    }

}

sub debugPrint {

    print @_ ;

}

sub GetCasTgt {

    #parameter 

    my($link, $uname, $pass) = @_;

    #command formulation

    my($cmd) = "curl -i -L -k -d \"username=$uname&password=$pass\" $link 2>/dev/null ";

    debugPrint("$cmd \n");

    my $tgt_response = qx/$cmd/;

    return undef if (!defined($tgt_response) || $tgt_response =~ /^\s*$/);      

    my @values =  split (/Location:/,$tgt_response);

    debugPrint($values [1]);

    my @val =  split(/Server/ , $values [1]);

    my @tgts = split (/\/v1\// , $val [0]);

    $tgt = $tgts[1];

    #remove the word "tickets" from $tgt

    $tgt =~ s/tickets\/(\w*)/$1/;

    debugPrint("\n\n=====> tgt is ${tgt}\n\n");

    return $tgt;

}

sub GetCasSt {

    my($url, $cas_link, $tgt) = @_;

    my ($cmd) = "curl -i -L -k -d \"service=$url\" $cas_link/$tgt 2>/dev/null";

    debugPrint("CMD: $cmd \n\n");

    my $ts_response = qx/$cmd/;

    debugPrint ("\n\n ====>st is ${ts_response} =======\n\n");

    my @ts = split(/ST/,$ts_response);

    my $ticket_service = "ST${ts[1]}";

    return $ticket_service;

}

sub ConnectToUrl {

    my($url, $ticket, $cookieFile) = @_;

    my($cmd) = "curl -c $cookieFile -i -L -k -H \"X-EMC-MAC-NO-TICKET-REDIRECT: TRUE\" -H \"X-EMC-MAC-USE-401: TRUE\" $url?ticket=$ticket  2>/dev/null";

    debugPrint ("======= From ConnecToUrl: $cmd ====\n\n");

    my @list = qx/$cmd/;

    my $line;

    foreach $line(@list) {

        debugPrint("$line");

        if ($line =~ /^HTTP\S*\s+(\d+)/) {

            return ($1, @list);

        } 

    }

    return (0, @list);

}

sub ConnectToUrlWithCookie {

    my($url, $cookieFile) = @_;

    my($command) = sprintf("curl -b %s -i -L -k -H \"X-EMC-MAC-NO-TICKET-REDIRECT: TRUE\" -H \"X-EMC-MAC-USE-401: TRUE\" %s 2>/dev/null", $cookieFile, $url);

    my @list = qx/$command/;

    debugPrint("===From ConnectToUrlWithCookie: $command===\n\n");

    #debugPrint(join("\n", @list));

    my($line);

    my($code) = 0;

    my($location) = "";

    foreach $line(@list) {

        debugPrint("$line");

        if ($line =~ /^HTTP\S*\s+(\d+)/) {

            $code = $1;

        } 

        if ($line =~ /Location:\s+([\S]+)/) {

            $location = $1;

        } 

    }

    return ($code, $location, @list);

}

sub Syntax {

    print "perl-curl-example -s 
    [-u 
    ] [-p <
     password>] \n";
    
   

    exit 1;

}

Regards

Conor

Hi,

We in support have a utility called PSCOLLECT utility which was deloped by a colleague in support.

PSCOLLECT uses ProSphere REST API to collect information from the SRM and SDA appliances. We in support usethis tool to troubleshoot.

This tool would pull back the inforamtion on Arrays,Host and switches discovered using REST API and the contents be displayed in HTMP and XML format.

To request this tool open a Chat with ProSphere Support.

This utility can be executed on a Windows host that can resolve DNS names of the ProSphere vApp appliances.

PSCOLLECT 0.9.6 is the latest build available.

To run pscollect, unzip pscollect_0.9.6.zip in a temp folder and execute pscollect.bat either on the command line or in the file explorer. When the program completes it places all the data collected within a zip file.

Within the pscollect_out.zip file created after the program runs you will have various information including arrays folder - information on discovered Symmetrix and Clariion arrays, switches folder - information on discovered physical/logical switches and hosts folder - information on discovered hosts.

The information will be in html and xml format.

The XML files can be converted to csv files for example using the tools available here http://xmlgrid.net/xml2text.html

This is example of a program pulling information from the ProSphere database using REST API

Regards

Conor

Hi Coron:

thanks for the kindly reply first.

I folowed your instruction here, but i did not get 401 response from Prosphere.

Sorry to upload my source code, just in case you can help to point out what i am doing wrong here.

Thanks a lot.

br

jigang

Hi Cornor

I also tried to send an http request with HttpClient.

Attached the log just FYI. you can see that the response i get from Prosphere after my first request is 302 , but not 401.

Thanks a lot for your patience.

br

jigang

I have the exact same problem, if I execute the powershell script from the wiki. Initially I have the ssl error which I fixed by importing the certificate to the trusted root section, then the $request contains the html for the login page.