Highlighted
jigang1
1 Copper

Java Rest Client to get topology data

<SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"><SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"> 

Hi

     i am trying to develop a java client to get topology(array,host,switch) data from prosphere with Rest API.  here is the url i am using

http://gsapsapp.209domain.emc.com/srm/topology/msa/types/PhysicalSwitch/instances,  but prosphere keep redirecting me to the login page even i put my cridential(username,password) in my http post message.

<SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"><SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;">     1. is there a demo java Rest client available?

<SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"><SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;">     2. what is the right process to get data?

<SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"><SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;">

<SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"><SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;">     thanks very much in advance.

<SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"><SPAN style="COLOR: #2a00ff; FONT-SIZE: 10pt" mcestyle="color: #2a00ff; font-size: 10pt;"></SPAN></SPAN> </P></SPAN></SPAN>

Tags (1)
0 Kudos
5 Replies
ConorLawlor
3 Argentium

Re: Java Rest Client to get topology data

Hi,

Download the EMC ProSphere REST API 1.6 from from the product page https://support.emc.com/products/15995 It describe CAS authentication for ProSphere

Review the CAS Authentication section and the following is detailed which show you examples of how to write batch files or Perl scripts examples so the request will not be blocked for login username / passwords details.

You will not be able to directly pass the Username and password using the URL will not work as CAS authentication is required. Using the browsers add-ons will allow you to enter the login/password details during the page request by following steps 1 to 8 below.

Some URIs require you to issue POST, PUT, and DELETE HTTP requests. To do this from a web browser, you need plugins for Internet Explorer and Firefox. The recommended plugins are:

__MCE_ITEM____MCE_ITEM__·         httpAnalyzer for Internet Explorer browsers

__MCE_ITEM____MCE_ITEM__·         Poster for Firefox and Chrome browsers

White paper called "ProSphere Integration: An overview of Rest APIs and Information Model" available on the product page which provides an overview of the ProSphere information architecture and REST APIs for ProSphere 1.0 but the ProSphere 1.6 REST API has been redesigned from the 1.0 version, providing a more robust API with increased functionality that allows programmers to extract ProSphere topology, performance, and alert data for integration into their storage environments.

CAS Authentication

The ProSphere 1.6 REST API is protected by the Central Authentication Service (CAS).

http://en.wikipedia.org/wiki/Central_Authentication_Service

All requests to the ProSphere REST interface must be authenticated, either by passing valid credentials in the request, or by passing in a cookie containing valid credentials.

The following basic steps describe CAS authentication in a ProSphere environment:

__MCE_ITEM__1.     Execute a GET against the target URL.

__MCE_ITEM__2.     Receive 401. Look at location: header.

__MCE_ITEM__3.     Do POST to that location. Include the username and password in body of that POST.

__MCE_ITEM__4.     Examine location header of the response. This is your session resource.

__MCE_ITEM__5.     Do a POST to the session resource location. The body of the POST must contain the URL you want to use.

__MCE_ITEM__6.     Take the contents of the response body, and add it as a query string argument to the URL you are trying to get.

__MCE_ITEM__7.     Execute the original GET request. Append the query string parameter ?ticket=. Include the cookies you are given.

__MCE_ITEM__8.     For additional requests, always resend the cookies received in step 7.

CAS Authentication Examples

The following batch file allows you to log in to ProSphere and submit a URL to examine. Note that the ProSphere server name is hard coded in this simple example.

@ECHO OFF
REM A script to do a RESTful CAS login
REM There is no error handling. 
REM
REM Usage:  curlcas.bat [username] [password] [URL-for-the-application-entry-point]
REM
REM For example:
REM 
REM curlcas.bat username mypassword https://ProSphereServerName/srm/topology/msa/types/StorageArray/instances 
REM
REM After that you can do any normal curl operations such as GETs, as long as you pass the one additional switch to read the cookies.txt file.  
REM 
REM curl -b cookies.txt -G https://ProSphereServerName/srm/topology/msa/types/StorageArray/instances
REM 
REM This will present the required CAS cookie to the application server, and curl just works as usual against a CAS protected site.
REM 
REM 
SETLOCAL
REM define the environment variable for where curl is installed.
SET MYCURLDIR=c:\curl
REM define the environment variable for the login command.  Note that the location of the CAS server is hard coded here.
REM This could/should be passed as an input environment variable, or perhaps as CLI argument %4.
SET LOGIN_CMD=%MYCURLDIR%\curl -s --insecure -i -POST https://ProSphereServerName/cas-server/v1/tickets --data username^^=%1 --data password^^=%2
REM
REM Do the login command and parse the response for the location header.
REM
FOR /F "usebackq eol= skip=3 tokens=1,2 delims= " %%i in (`%LOGIN_CMD%`) DO IF /I [%%i] == [Location:] SET PLACE_TO_POST=%%j
REM Create the command to get a Service Ticket from the RESTful interface for CAS.  Parse the return result for the ticket.
REM
SET GET_ST_CMD=%MYCURLDIR%\curl -s --insecure --data service^^=%3 -POST %PLACE_TO_POST%
FOR /F "usebackq tokens=1 delims=" %%i in (`%GET_ST_CMD%`) DO SET SERVICE_TICKET=%%i
REM
REM Present the ticket to the target service.  Notice you need to store the newly established session via the cookies file. 
SET FINAL_COMMAND=%MYCURLDIR%\curl -s --insecure -c cookies.txt -G %3?ticket=%SERVICE_TICKET%
%FINAL_COMMAND% > NUL
REM
REM All done.  Now you can run any curl command, passing in the same cookies file.
REM such as doing:
curl -k -b cookies.txt -G %3
REM
ENDLOCAL

Perl example

The following example is coded in Perl. This script automatically determines the location of the CAS server based on the URL passed in the original GET request.

#!/usr/bin/perl
#
# * Copyright 2009, EMC Corporation ("EMC").
# * All Rights Reserved
# *
# * UNPUBLISHED CONFIDENTIAL AND PROPRIETARY PROPERTY OF EMC. The copyright
# * notice above does not evidence any actual or intended publication of this
# * software. Disclosure and dissemination are pursuant to separate agreements.
# * Unauthorized use, distribution or dissemination are strictly prohibited.
#
# Purpose: reload.monitor will query the registry service and rewrite mon  
#          configuration files if the registry contents have changed
#
# reload.monitor -u <service> <host1> [<hosts...>]
#
# -u <service url> :  url of the registry service
# -e <mon.cf>      :  location of the mon config file to write
# -d               :  enable debug
#
use URI;
use Getopt::Std;
use English;
use Data::Dumper;
use XML::Simple;
use File::Compare;
sub httpGET;
$opt_s = undef;
$opt_u = undef;
$opt_p = undef;
getopts ("s:u:p:");
$uname=$opt_u;
$pass=$opt_p;
if (!defined($uname)) {
    $uname = "appadmin";
}
if (!defined($pass)) {
    $pass = "Changeme1!";
}
my $cookieFile = "my_cookie.txt";
# this option has to be there
my $serviceLink = $opt_s;
Syntax() if (!defined($serviceLink));
$url = $serviceLink;
my ($result, $content) = checkURL($url, $cookieFile);
#  Functions
#####################################################################################
sub checkURL {
    my($url, $cookieFile) = @_;
    #Constants
    debugPrint("url: $url \n");
    debugPrint("cookieFile: $cookieFile\n");
    # assume to have a valid cookie to run first
    my($code, $location, @content) = ConnectToUrlWithCookie($url, $cookieFile);
    debugPrint("code from ConnectToURLWithCookie: $code");
    if ($code == 401) {
   
       # no cookie, or cookie expired
       #a)get CAS URI from 401 return code
       #b)get "internal" credentials from lockbox
       #c)get TGT, ST
       #d)try to connect the service URL again
    
       $cas_link = $location if (!defined($cas_link));
       # Now it is 401 (unauthorized access) 
       debugPrint("cas_link: $cas_link\n");
    
       if (!defined($uname) || !defined($pass)){
           return(1, "Cannot get username/password from lockbox \n");
       }
       debugPrint (" ============= GetCasTGT ============\n");
       my $tgt = GetCasTgt($cas_link, $uname, $pass);
       if (!defined($tgt) || $tgt =~ /^\s*$/) {
           return (1, "Cannot get Ticket Granting Ticket from CAS\n");
       }
       debugPrint ("TGT = $tgt \n");
       debugPrint (" ============= GetCasST ============\n");
       my $st = GetCasSt($url, $cas_link, $tgt);
       if (!defined($st) || ($st =~ /^\s$/)) {
           return (1, "Cannot get Service Ticket from CAS\n");
       }
       debugPrint ("ST => $st \n");
       ($code, @content) = ConnectToUrl($url, $st, $cookieFile);
       debugPrint ("Connect to URL code $code \n");
       ConnectToUrlWithCookie($url, $cookieFile);
    }
}
sub debugPrint {
    print @_ ;
}
sub GetCasTgt {
    #parameter 
    my($link, $uname, $pass) = @_;
    
    #command formulation
    my($cmd) = "curl -i -L -k -d \"username=$uname&password=$pass\" $link 2>/dev/null ";
    debugPrint("$cmd \n");
    my $tgt_response = qx/$cmd/;
    return undef if (!defined($tgt_response) || $tgt_response =~ /^\s*$/);      
    my @values =  split (/Location:/,$tgt_response);
    debugPrint($values [1]);
    my @val =  split(/Server/ , $values [1]);
    my @tgts = split (/\/v1\// , $val [0]);
    $tgt = $tgts[1];
    #remove the word "tickets" from $tgt
    $tgt =~ s/tickets\/(\w*)/$1/;
    debugPrint("\n\n=====> tgt is ${tgt}\n\n");
   
    return $tgt;
}
sub GetCasSt {
    my($url, $cas_link, $tgt) = @_;
    
    my ($cmd) = "curl -i -L -k -d \"service=$url\" $cas_link/$tgt 2>/dev/null";
    debugPrint("CMD: $cmd \n\n");
    my $ts_response = qx/$cmd/;
    
    debugPrint ("\n\n ====>st is ${ts_response} =======\n\n");
    my @ts = split(/ST/,$ts_response);
    my $ticket_service = "ST${ts[1]}";
    return $ticket_service;
}
sub ConnectToUrl {
    my($url, $ticket, $cookieFile) = @_;
    my($cmd) = "curl -c $cookieFile -i -L -k -H \"X-EMC-MAC-NO-TICKET-REDIRECT: TRUE\" -H \"X-EMC-MAC-USE-401: TRUE\" $url?ticket=$ticket  2>/dev/null";
    debugPrint ("======= From ConnecToUrl: $cmd ====\n\n");
    my @list = qx/$cmd/;
    my $line;
        
    foreach $line(@list) {
        debugPrint("$line");
        if ($line =~ /^HTTP\S*\s+(\d+)/) {
            return ($1, @list);
        } 
    }
     
    return (0, @list);
}
sub ConnectToUrlWithCookie {
    my($url, $cookieFile) = @_;
    my($command) = sprintf("curl -b %s -i -L -k -H \"X-EMC-MAC-NO-TICKET-REDIRECT: TRUE\" -H \"X-EMC-MAC-USE-401: TRUE\" %s 2>/dev/null", $cookieFile, $url);
    my @list = qx/$command/;
     
    debugPrint("===From ConnectToUrlWithCookie: $command===\n\n");
    #debugPrint(join("\n", @list));
    
    my($line);
    my($code) = 0;
    my($location) = "";
    foreach $line(@list) {
        debugPrint("$line");
        if ($line =~ /^HTTP\S*\s+(\d+)/) {
            $code = $1;
        } 
        if ($line =~ /Location:\s+([\S]+)/) {
            $location = $1;
        } 
    }
    return ($code, $location, @list);
    
}
sub Syntax {
    print "perl-curl-example -s <servicelink> [-u <username>] [-p <password>] \n";
    exit 1;
}

Regards

Conor

0 Kudos
ConorLawlor
3 Argentium

Re: Java Rest Client to get topology data

Hi,

We in support have a utility called PSCOLLECT utility which was deloped by a colleague in support.

PSCOLLECT uses ProSphere REST API to collect information from the SRM and SDA appliances. We in support usethis tool to troubleshoot.

This tool would pull back the inforamtion on Arrays,Host and switches discovered using REST API and the contents be displayed in HTMP and XML format.

To request this tool open a Chat with ProSphere Support.

This utility can be executed on a Windows host that can resolve DNS names of the ProSphere vApp appliances.

PSCOLLECT 0.9.6 is the latest build available.

To run pscollect, unzip pscollect_0.9.6.zip in a temp folder and execute pscollect.bat either on the command line or in the file explorer. When the program completes it places all the data collected within a zip file.

Within the pscollect_out.zip file created after the program runs you will have various information including arrays folder - information on discovered Symmetrix and Clariion arrays, switches folder - information on discovered physical/logical switches and hosts folder - information on discovered hosts.

The information will be in html and xml format.

The XML files can be converted to csv files for example using the tools available here http://xmlgrid.net/xml2text.html

This is example of a program pulling information from the ProSphere database using REST API

Regards

Conor

0 Kudos
jigang1
1 Copper

Re: Java Rest Client to get topology data

Hi Coron:

thanks for the kindly reply first.

I folowed your instruction here, but i did not get 401 response from Prosphere.

Sorry to upload my source code, just in case you can help to point out what i am doing wrong here.

Thanks a lot.

br

jigang

0 Kudos
jigang1
1 Copper

Re: Java Rest Client to get topology data

Hi Cornor

I also tried to send an http request with HttpClient.

Attached the log just FYI. you can see that the response i get from Prosphere after my first request is 302 , but not 401.

Thanks a lot for your patience.

br

jigang

0 Kudos
poneros
1 Copper

Re: Java Rest Client to get topology data

I have the exact same problem, if I execute the powershell script from the wiki. Initially I have the ssl error which I fixed by importing the certificate to the trusted root section, then the $request contains the html for the login page.

0 Kudos