Systems Management General

Last reply by 12-05-2022 Solved
Start a Discussion
2 Bronze
2 Bronze
33470

iDRAC 6 virtual console does not work with Java 8

After updating iDRAC6 FW and the LCC FW, it seems now that Java wanted to update too so now it's version 8.0.251, and now I can't get the virtual console to connect, because Java blocks it, I even added my server IP to Java's exception list, but nope, Java still blocks it, I don't want to remove v8, and re-install v7, because a lot of apps that use Java, insist on using v8, is there a way to force Java 8 to accept the self signed certificates that iDRAC creates every time the console is launched?

It seems now that Oracle in their (unknown wisdom) have decided that self signed certificates are taboo, and should NEVER be used, which forces us to buy an SSL certificate from a reputable cert service even though we're only using local LAN access only, IE, iDRAC is only accessible from within the LAN, as nothing is being forwarded to the iDRAC dedicated IP.

Java 7 had the option in the security tab of 'Medium' settings which allows the virtual console to run, but Java 8 has removed that setting, and now it's only high or very high.

Solution (1)

Accepted Solutions
33365

OK, I just found someone that has the exact same problem, and he fixed it, I'll explain what he did, for anyone that has the same problem with using the iDRAC 6 Enterprise Virtual Console with Java 8.

The problem is that Oracle in their (unknown wisdom) decided that self signed certificates are taboo, and must NEVER be used, thus basically forcing people to go out and buy a signed certificate from an accredited authority, but there is a way to get round the Java security blocking the virtual console.

First, if you try to launch the console you'll get an error that Java has blocked the certificate, so in that case you'll need to open the Java control panel which in windows 10 is in the Windows control panel under Java or Java(32), then when the Java control panel is open, click the 'Security' tab, then at the bottom, click 'Edit Site List', then click 'Add' then enter your server's LAN IP address, then click OK, then click OK again, but if you try to use the console, you'll eventually get a 'Connection Failed' error, this is because of the Java crypto algorithm used, and Dell uses RC4, so this is easily remedied in the java.security file.

Second, open a command prompt with elevated privileges, IE cmd with admin access, by opening the windows search then type cmd and right click the cmd line and select 'Run as administrator', then navigate to the java security file which in Windows 10 is at:-

C:\Program Files (x86)\Java\jre1.8.0_251\lib\security

In the command prompt type cd "C:\Program Files (x86)\Java\jre1.8.0_251\lib\security"   Don't forget the quotes, otherwise windows will give you an error.

If you're reading this when there's been an update, then replace jre1.8.0.251 with the version number you're currently using.

Third, in the command prompt type this:-

notepad java.security

Press enter, then press CTRL-F and look for jdk.tls

When it's found look for the line:-

jdk.tls.disabledAlgorithms=SSLv3, DES, RC4, MD5withRSA, DH keySize < 1024, \
EC keySize < 224, 3DES_EDE_CBC, anon, NULL

Delete the bit that has RC4, (including the comma) so that it looks like this:-

jdk.tls.disabledAlgorithms=SSLv3, DES, MD5withRSA, DH keySize < 1024, \
EC keySize < 224, 3DES_EDE_CBC, anon, NULL

Save it and then you can successfully connect to your server using the iDRAC 6 Enterprise Virtual Console.

I just tried it and it works.

View solution in original post

Replies (10)
33429

Hi,

 

Could you check in virtual console, if you are able to switch the plug in type to native or HTML5? 

 

Alternative, have you tried these steps on a previous post: https://dell.to/2ALrh5q


DELL-Joey C
Social Media and Communities Professional
Dell Technologies | Enterprise Support Services
#IWork4Dell

Did I answer your query? Please click on ‘Accept as Solution’. ‘Kudo’ the posts you like!

33380

Sorry about the late reply.

No, the virtual console has only 'Native' or 'Java', not HMTL5, and both do the same.

I looked that link, but it was a file association problem with that one, but in my case, Java 8 blocks the certificate, and when it says 'connecting to virtual console' it's then that Java shows something like "due to your security setting, the certificate has been blocked, because it is not from a trusted authority", and the security setting is related to the Java security settings, which in Java 8, they have removed the 'Medium' security setting, and now only have High and Very High.  In Java 7, it was the 'Medium' settings that allowed the certificate, so now, until it's fixed, I had to revert back to Java 7.

33366

OK, I just found someone that has the exact same problem, and he fixed it, I'll explain what he did, for anyone that has the same problem with using the iDRAC 6 Enterprise Virtual Console with Java 8.

The problem is that Oracle in their (unknown wisdom) decided that self signed certificates are taboo, and must NEVER be used, thus basically forcing people to go out and buy a signed certificate from an accredited authority, but there is a way to get round the Java security blocking the virtual console.

First, if you try to launch the console you'll get an error that Java has blocked the certificate, so in that case you'll need to open the Java control panel which in windows 10 is in the Windows control panel under Java or Java(32), then when the Java control panel is open, click the 'Security' tab, then at the bottom, click 'Edit Site List', then click 'Add' then enter your server's LAN IP address, then click OK, then click OK again, but if you try to use the console, you'll eventually get a 'Connection Failed' error, this is because of the Java crypto algorithm used, and Dell uses RC4, so this is easily remedied in the java.security file.

Second, open a command prompt with elevated privileges, IE cmd with admin access, by opening the windows search then type cmd and right click the cmd line and select 'Run as administrator', then navigate to the java security file which in Windows 10 is at:-

C:\Program Files (x86)\Java\jre1.8.0_251\lib\security

In the command prompt type cd "C:\Program Files (x86)\Java\jre1.8.0_251\lib\security"   Don't forget the quotes, otherwise windows will give you an error.

If you're reading this when there's been an update, then replace jre1.8.0.251 with the version number you're currently using.

Third, in the command prompt type this:-

notepad java.security

Press enter, then press CTRL-F and look for jdk.tls

When it's found look for the line:-

jdk.tls.disabledAlgorithms=SSLv3, DES, RC4, MD5withRSA, DH keySize < 1024, \
EC keySize < 224, 3DES_EDE_CBC, anon, NULL

Delete the bit that has RC4, (including the comma) so that it looks like this:-

jdk.tls.disabledAlgorithms=SSLv3, DES, MD5withRSA, DH keySize < 1024, \
EC keySize < 224, 3DES_EDE_CBC, anon, NULL

Save it and then you can successfully connect to your server using the iDRAC 6 Enterprise Virtual Console.

I just tried it and it works.

33168

One minor suggestion:

"The problem is that Oracle in their (unknown wisdom) decided that self signed certificates are taboo, and must NEVER be used, thus basically forcing people to go out and buy a signed certificate from an accredited authority"

From a security point of view, self-signed certificates really do pose a threat. In a corporate setting, however small, it could pay off to put a little time into building your own CA. That way, you can give out trusted certificates to all your systems "for free". Of course, there's no such thing as free And learning to build a PKI is a whole different kettle of fish!

27414

EDIT: Could have been expired java file download and after changing in console settings from native to java and redownloading jave file it's now working.

It solve my problem partially on R710 2.92 iDRAC firmware. I got the Failed to connect error, removed RC4 from java.security but now am getting "Login failed, possibly due to slow network connection. Please try again later". I'm on a 1GB LAN so a slow connection is not the case. Thanks for the solution anyway. Am looking further to solve the new problem.

2 Bronze
2 Bronze
14305

Newer versions of Java 8 appear to ignore those security patches.

I am trying to find a version of Java 6 or Java 7 to install on my PC so I can access the Virtual Console on my R610s and R620s

2 Bronze
2 Bronze
13501

In agreement with RobR, the usalabs solution here is no longer working with Java 8 patch level 321.

I have an iDRAC7 on a Dell PowerEdge T320. Such misery Java is. May it RIP (all versions). 

Old versions of Java available at https://www.oracle.com/java/technologies/javase/javase7-archive-downloads.html You must create an account to download.

I am trying Java 7 patch 80.

Using Control Panel Java applet, modified all settings to the least restrictive possible. Logged off and closed browser window to iDRAC. Downloaded a new viewer.jnlp file. In Windows File Exporer > Properties > Unblock. The C:\Program Files\Java\jre7\lib\security>notepad java.security contents is "#jdk.tls.disabledAlgorithms=SSLv3" (commented out ALL disablement). 

Java prompts to trust the certificate from the iDRAC. I view it, see that it is SHA1withRSA, self-signed.

Error message is "The viewer has terminated. Reason: The network connection has been dropped."

In the Java Console:

Supported protocols: [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]

Enabled protocols: [SSLv3, TLSv1]

 

Still no luck. I'm posting this here now, will continue  to Google for a solution. But I'm stimied. Going to have to get out physical keyboard, mouse, and video display I guess. Cripes, what a nightmare.

 

11018

I have the most recent version of Java in March, 2022 and the above does almost work.

 

This works:

Only works in Firefox browser - I gave up on the others but it might still be possible.

Remove TLS1 from the mentioned line - I removed RC4 also, just to be sure. 
Set browser to accept TLS1

Add your idrac domain name INCLUDING HTTPS:// to the java server allowed list
Configured .jnlp to automatically open with Java web launcher (otherwise you get a connection timeout).

I'm now administering my old dell r610

 

2 Bronze
2 Bronze
1028

Another option that works on Edge

Install the JNLP fixer from the Chrome webstore

using chcolately, install an old and insecure version of java with command

choco install jre6

 

Run edge as administrator and download the jnlp file, select KEEP and open it with C:\Program Files (x86)\Java\jre6\bin\javaws.exe

 

Tested with Edge 107.0.1418.42 64bit on Windows 21H2 64bit and a DELL R510 with iDRAC 6 2.91 (Build 02)

 

Latest Solutions
Top Contributor