AppSync：文件系统映射失败，并显示“HeapDumpOnOutOfMemoryError”

AppSync 作业挂起，后续作业无法运行。

AppSync 服务器日志（AppSync 服务器上的 C:\EMC\AppSync\jboss\logs\server.log）中，我们看到 AppSync 服务器服务在此处重新启动...

07-30-2016 01:10:13.494 INFO [MSC service thread 1-1] [org.jboss.as.naming] [] [] JBAS011802: Starting Naming Service
07-30-2016 01:10:14.823 INFO [MSC service thread 1-7] [org.jboss.ws.common.management.AbstractServerConfig] [] [] JBoss Web Services - Stack CXF Server 4.0.2.GA

在 AppSync 执行日志（AppSync 服务器上的 C:\EMC\AppSync\jboss\logs\executive.log）中，我们看到以下内存不足错误：

4098664.623： [Full GC
 [PSYoungGen: 384064K->384062K(404032K)] [ParOldGen: 1398143K->1398143K(1398144K)] 1782207K->1782205K(1802176K) [PSPermGen: 157954K->157954K(158272K)], 1.4650114 secs] [Times: user=5.06 sys=0.00, real=1.47 secs]
Shutdown failed: Timed out waiting for signal from JVM.
Enqueue Event 'jvm_kill'
4098666.090: [Full GC
JVM did not exit on request, terminated
Enqueue Event 'jvm_killed'
Enqueue Event 'jvm_stopped'
socket read failed. （远程主机强制关闭了现有连接。 (0x2746))
Closing backend socket.
Preparing to restart with mode 4.
JVM was only running for -547624 seconds leading to a failed restart count of 1.
Enqueue Event 'jvm_failed_invocation'
Waiting 5 seconds before launching another JVM.
server listening on port 32000.
Ping settings: wrapper.ping.interval=5, wrapper.ping.interval.logged=1, wrapper.ping.timeout=300
Enqueue Event 'jvm_restart'
Enqueue Event 'jvm_prelaunch'
Launching a JVM...
Command: "_jre\bin\java" -server -Xms1024m -Xmx2048m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintCommandLineFlags -XX:+UseCompressedOops -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=false -Dorg.jboss.resolver.warning=true -Dorg.tanukisoftware.wrapper.WrapperManager.mbean=false -Djboss.modules.system.pkgs=org.jboss.byteman -Djboss.server.default.config=standalone.xml -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dorg.jboss.boot.log.file=logs/boot.log -Dlogging.configuration=file:standalone/configuration/logging.properties -Djboss.server.log.dir=logs -Dappsync.server=SVRAPPSYNC.PHYSIOCORP.COM -Dfile.encoding=UTF8 -Duser.language=en -Djava.library.path="executive;../CST/builds/win_vc100_x64_r/bin" -classpath "executive/wrapper.jar;./jboss-modules.jar" -Dwrapper.key="g6PkoEa65igOyT26W3eDRDQ6_Kektd7Q" -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.debug="TRUE" -Dwrapper.pid=3408 -Dwrapper.version="3.5.14-pro" -Dwrapper.native_library="wrapper" -Dwrapper.service="TRUE" -Dwrapper.cpu.timeout="10" -Dwrapper.jvmid=2 -Dwrapper.lang.domain=wrapper -Dwrapper.lang.folder=../lang org.tanukisoftware.wrapper.WrapperJarApp jboss-modules.jar -mp modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=.
JVM started (PID=7788)
Enqueue Event 'jvm_start'
-XX:+HeapDumpOnOutOfMemoryError -XX:InitialHeapSize=1073741824 -XX:MaxHeapSize=2147483648 -XX:MaxPermSize=536870912 -XX:+PrintCommandLineFlags -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedOops -XX:-UseLargePagesIndividualAllocation -XX:+UseParallelGC
WrapperManager Debug: WrapperManager class initialized by thread: main   Using classloader: sun.misc.Launcher$AppClassLoader@78d5c653
WrapperManager: 正在初始化...
WrapperManager Debug: JVM #2
WrapperManager Debug: Running a 64-bit JVM.
WrapperManager Debug: org.tanukisoftware.wrapper.WrapperManager package information:
WrapperManager Debug:   Implementation Title: org.tanukisoftware.wrapper
WrapperManager Debug:   实施供应商：Tanuki Software，Ltd.
WrapperManager Debug:   实施版本： 3.5.14
WrapperManager Debug:   密封？： False
WrapperManager Debug: org.tanukisoftware.wrapper.WrapperManager protection domain:
WrapperManager Debug:   Location: file:/C:/EMC/AppSync/jboss/executive/wrapper.jar
WrapperManager Debug:     大小： 117,262
WrapperManager Debug:     MD5： 3a16b23744071559f42f26c48b36e756
WrapperManager Debug: 注册关机挂机
WrapperManager Debug: Using wrapper
WrapperManager Debug: 加载本机库。  有多个可能的文件名，将使用第一个找到的文件名。  加载非现有文件时出错是正常的，只有在它们全部失败时才会出现问题。
WrapperManager Debug:   Attempt to load native library with name: wrapper-windows-x86-64.dll  Result: no wrapper-windows-x86-64 in java.library.path
WrapperManager Debug:   Attempt to load native library with name: wrapper.dll  Result: 成功！
WrapperManager Debug:   已成功加载本机库。
WrapperManager Debug: 已加载本地化资源。
WrapperManager Debug: 调用本机初始化方法。
WrapperJNI Debug: 正在初始化 WrapperManager 本机库。
WrapperJNI Debug: Java 可执行文件： C:\EMC\AppSync\jboss\_jre\bin\java.exe
WrapperJNI Debug: Native Library: C:\EMC\AppSync\jboss\executive\wrapper.dll
WrapperJNI Debug: Windows 版本： 6.2.9200
WrapperManager Debug: Java 版本： 1.7.0_17-b02 Java HotSpot(TM) 64-Bit Server VM
WrapperManager Debug: Java VM 供应商： Oracle Corporation
WrapperManager Debug: 操作系统名称       ： Windows Server 2012
WrapperManager Debug: OS Arch        : amd64
WrapperManager Debug:
WrapperManager Debug: 控制事件监视器线程已启动。
WrapperManager Debug: 启动运行器线程已启动。
WrapperJarApp Debug: Jar Main-Class: org.jboss.modules.Main
WrapperJarApp Debug: Jar 类路径：未指定。
WrapperJarApp Debug:     Classpath[0]=file:C:/EMC/AppSync/jboss/jboss-modules.jar
WrapperManager Debug: WrapperManager.start(org.tanukisoftware.wrapper.WrapperJarApp@5bfa19ca, args["-mp", "modules", "-jaxpmodule", "javax.xml.jaxp-provider", "org.jboss.as.standalone", "-Djboss.home.dir=."]) called by thread: main
WrapperManager Debug: 通信运行器线程已启动。
WrapperManager Debug: Open socket to wrapper...Wrapper-Connection
WrapperManager Debug: Opened Socket from 31,000 to 32,000
WrapperManager Debug: Send a packet KEY : g6PkoEa65igOyT26W3eDRDQ6_Kektd7Q
WrapperManager Debug: handleBackend()
accepted a socket from 127.0.0.1 on port 31000
closing backend server.
read a packet KEY : g6PkoEa65igOyT26W3eDRDQ6_Kektd7Q
Got key from JVM: g6PkoEa65igOyT26W3eDRDQ6_Kektd7Q
send a packet LOW_LOG_LEVEL : 1
send a packet LOGFILE : logs\executive.log
send a packet PROPERTIES : (Property Values)
Start Application.
send a packet START : start
WrapperManager Debug: Received a packet LOW_LOG_LEVEL : 1
WrapperManager Debug: LowLogLevel from Wrapper is 1
WrapperManager 调试： Received a packet LOGFILE : logs\executive.log
WrapperManager Debug: 接收到的数据包属性: (Property Values)
WrapperManager Debug: Received a packet START : start
WrapperManager Debug: Java 临时目录： C:\Windows\TEMP
WrapperManager Debug: calling WrapperListener.start()
WrapperManager Debug: WrapperListener.start runner thread started.
WrapperJarApp Debug: start(args) Will wait up to 2 seconds for the main method to complete.
WrapperJarApp Debug: invoking main method
WrapperManager Debug: Send a packet START_PENDING : 5000
read a packet START_PENDING : 5000
JVM signaled a start pending with waitHint of 5000 millis.
WrapperManager Debug: Send a packet START_PENDING : 5000
WrapperJarApp Debug: start(args) end.  Main Completed=false, exitCode=null
WrapperManager Debug: returned from WrapperListener.start()
WrapperManager Debug: Send a packet STARTED :
WrapperManager Debug: WrapperListener.start runner thread stopped.
WrapperManager Debug: Startup runner thread stopped.
read a packet START_PENDING : 5000
JVM signaled a start pending with waitHint of 5000 millis.
read a packet STARTED :
JVM signaled that it was started.
Enqueue Event 'jvm_started'
send a packet PING : ping
WrapperManager Debug: Received a packet PING : ping
WrapperManager Debug: Send a packet PING : ping
read a packet PING : ping
INFO  (WrapperJarAppMain) 2016-07-30 01:10:02,400 [org.jboss.modules] Module JBoss Modules version 1.1.1.GA
send a packet PING : ping
WrapperManager Debug: Received a packet PING : ping
WrapperManager Debug: Send a packet PING : ping
read a packet PING : ping
INFO  (WrapperJarAppMain) 2016-07-30 01:10:05,072 [org.jboss.msc] ServiceLogger_$logger JBoss MSC version 1.0.2.GA
INFO  (MSC service thread 1-4) 2016-07-30 01:10:05,151 [org.jboss.as] ApplicationServerService JBAS015899: JBoss AS 7.1.1.Final "Brontes" starting
WrapperJarApp Debug: main method completed
send a packet PING : ping
WrapperManager Debug: Received a packet PING : ping
WrapperManager Debug: Send a packet PING : ping
read a packet PING : ping
INFO  (MSC service thread 1-3) 2016-07-30 01:10:11,057 [org.xnio] Xnio XNIO Version 3.0.3.GA
INFO  (Controller Boot Thread) 2016-07-30 01:10:11,057 [org.jboss.as.server] HttpManagementAddHandler JBAS015888: Creating http management service using socket-binding (management-http)
INFO  (MSC service thread 1-3) 2016-07-30 01:10:11,213 [org.xnio.nio] NioXnio XNIO NIO Implementation Version 3.0.3.GA
INFO  (ServerService Thread Pool -- 25) 2016-07-30 01:10:11,651 [org.jboss.as.configadmin] ConfigAdminAdd JBAS016200: Activating ConfigAdmin Subsystem
21.020: [GC [PSYoungGen： 262208K->21745K(305856K)] 262263K->21808K(1004928K), 0.0279106 secs] [Times: user=0.05 sys=0.03, real=0.03 secs]
21.051: [GC [PSYoungGen：82212K->21713K(305856K)] 82275K->21848K(1004928K), 0.0207127 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
INFO  (ServerService Thread Pool -- 30) 2016-07-30 01:10:11,713 [org.jboss.as.clustering.infinispan] InfinispanSubsystemAdd JBAS010280: Activating Infinispan subsystem.
INFO  (ServerService Thread Pool -- 46) 2016-07-30 01:10:

映射操作正在进行时，由于内存不足问题，AppSync Server 服务重新启动。这导致 AppSync 内部数据库中出现了过时的映射条目。

解决方案：

该修复将防止遇到过时条目，从而防止系统挂起并确保后续作业能够正常运行。
此问题已在 AppSync 2.2.3.1 中得到解决。