weblogic连接池重置（Connection reset）问题解决方案

共计 36719 个字符，预计需要花费 92 分钟才能阅读完成。

业务系统在高峰时期报错，需要对 weblogic 的日志进行分析，查看问题出现的原因以及解决方案。
在 weblogic 的 AdminServer.log 中的报错信息如下：

####<2017-8-8 下午 07 时 29 分 43 秒 CST> <Warning> <JDBC> <scyy1> <AdminServer> <pool-4514-thread-165> <<anonymous>> <> <> <1502191783426> <BEA-001129> <Received exception while creating connection for pool “CAMSCORE”: IO 错误: Connection reset.>
####<2017-8-8 下午 07 时 29 分 43 秒 CST> <Warning> <JDBC> <scyy1> <AdminServer> <pool-4514-thread-252> <<anonymous>> <> <> <1502191783427> <BEA-001129> <Received exception while creating connection for pool “CAMSCORE”: IO 错误: Connection reset.>
####<2017-8-8 下午 07 时 29 分 43 秒 CST> <Warning> <JDBC> <scyy1> <AdminServer> <pool-4514-thread-243> <<anonymous>> <> <> <1502191783428> <BEA-001129> <Received exception while creating connection for pool “CAMSCORE”: IO 错误: Connection reset.>
####<2017-8-8 下午 07 时 29 分 43 秒 CST> <Warning> <JDBC> <scyy1> <AdminServer> <pool-4514-thread-55> <<anonymous>> <> <> <1502191783428> <BEA-001129> <Received exception while creating connection for pool “CAMSCORE”: IO 错误: Connection reset.>
####<2017-8-8 下午 07 时 29 分 43 秒 CST> <Info> <JDBC> <scyy1> <AdminServer> <pool-4514-thread-55> <<anonymous>> <> <> <1502191783429> <BEA-001156> <Stack trace associated with message 001129 follows:
java.sql.SQLRecoverableException: IO 错误: Connection reset
at Oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:467)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:357)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:239)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1310)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1227)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:451)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:343)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:330)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:469)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:363)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:125)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:158)
at weblogic.jdbc.pool.Driver.connect(Driver.java:132)
at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
at weblogic.jdbc.jts.Driver.connect(Driver.java:127)
at weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
at com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.update(SourceFile:167)
at com.bocom.cams.core.base.commons.BaseDao.update(BaseDao.java:37)
at com.bocom.cams.core.base.commons.PersistableDao.update(PersistableDao.java:46)
at com.bocom.cams.core.base.commons.PersistableService.modify(PersistableService.java:101)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:499)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
at com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused By: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at oracle.net.ns.DataPacket.send(DataPacket.java:210)
at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:230)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:312)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:404)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:385)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:357)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:239)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1310)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1227)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:451)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:343)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:330)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:469)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:363)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:125)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:158)
at weblogic.jdbc.pool.Driver.connect(Driver.java:132)
at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
at weblogic.jdbc.jts.Driver.connect(Driver.java:127)
at weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
at com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.update(SourceFile:167)
at com.bocom.cams.core.base.commons.BaseDao.update(BaseDao.java:37)
at com.bocom.cams.core.base.commons.PersistableDao.update(PersistableDao.java:46)
at com.bocom.cams.core.base.commons.PersistableService.modify(PersistableService.java:101)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:499)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
at com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
>
####<2017-8-8 下午 07 时 29 分 43 秒 CST> <Info> <JDBC> <scyy1> <AdminServer> <pool-4514-thread-165> <<anonymous>> <> <> <1502191783427> <BEA-001156> <Stack trace associated with message 001129 follows:
java.sql.SQLRecoverableException: IO 错误: Connection reset
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:467)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:357)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:239)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1310)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1227)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:451)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:343)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:330)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:469)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:363)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:125)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:158)
at weblogic.jdbc.pool.Driver.connect(Driver.java:132)
at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
at weblogic.jdbc.jts.Driver.connect(Driver.java:127)
at weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
at com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.queryForList(SourceFile:108)
at com.bocom.cams.core.base.commons.BaseDao.find(BaseDao.java:60)
at com.bocom.cams.core.base.commons.PersistableDao.findByBizKeys(PersistableDao.java:76)
at com.bocom.cams.core.base.commons.PersistableService.queryByBizKeys(PersistableService.java:192)
at com.bocom.cams.core.base.commons.PersistableService.getByBizKeysFromDataBase(PersistableService.java:321)
at com.bocom.cams.core.base.commons.PersistableService.getByBizKeys(PersistableService.java:223)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.calcTypeOfFixedtimeProductInts(InterestCalculateService.java:1388)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:371)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
at com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused By: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at oracle.net.ns.DataPacket.send(DataPacket.java:210)
at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:230)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:312)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOSESSKEY(T4CTTIoauthenticate.java:404)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:385)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.makeConnection(ConnectionEnvFactory.java:357)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:239)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1310)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1227)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:451)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:343)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:330)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:469)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:363)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:125)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:158)
at weblogic.jdbc.pool.Driver.connect(Driver.java:132)
at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
at weblogic.jdbc.jts.Driver.connect(Driver.java:127)
at weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
at com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
at com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.queryForList(SourceFile:108)
at com.bocom.cams.core.base.commons.BaseDao.find(BaseDao.java:60)
at com.bocom.cams.core.base.commons.PersistableDao.findByBizKeys(PersistableDao.java:76)
at com.bocom.cams.core.base.commons.PersistableService.queryByBizKeys(PersistableService.java:192)
at com.bocom.cams.core.base.commons.PersistableService.getByBizKeysFromDataBase(PersistableService.java:321)
at com.bocom.cams.core.base.commons.PersistableService.getByBizKeys(PersistableService.java:223)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.calcTypeOfFixedtimeProductInts(InterestCalculateService.java:1388)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:371)
at com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
at com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
>

在应用日志中的报错信息如下：

19:31:43.633 [hostName:scyy1] [pool-4514-thread-243] [2017080900000020] ERROR 报错信息如下：weblogic.jdbc.extensions.ConnectionDeadSQLException: weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO 错误: Connection reset
weblogic.jdbc.common.internal.JDBCUtil.wrapAndThrowResourceException(JDBCUtil.java:250)
weblogic.jdbc.pool.Driver.connect(Driver.java:142)
weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
weblogic.jdbc.jts.Driver.connect(Driver.java:127)
weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.queryForList(SourceFile:108)
com.bocom.cams.core.base.commons.BaseDao.find(BaseDao.java:60)
com.bocom.cams.core.base.commons.PersistableDao.findByBizKeys(PersistableDao.java:76)
com.bocom.cams.core.base.commons.PersistableService.queryByBizKeys(PersistableService.java:192)
com.bocom.cams.core.base.commons.PersistableService.getByBizKeysFromDataBase(PersistableService.java:321)
com.bocom.cams.core.base.commons.PersistableService.getByBizKeys(PersistableService.java:223)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:248)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
java.lang.Thread.run(Thread.java:662)
19:31:43.633 [hostName:scyy1] [pool-4514-thread-260] [2017080900000020] ERROR 报错信息如下：weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CAMSCORE is Suspended, cannot allocate resources to applications..
weblogic.jdbc.common.internal.JDBCUtil.wrapAndThrowResourceException(JDBCUtil.java:251)
weblogic.jdbc.pool.Driver.connect(Driver.java:142)
weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
weblogic.jdbc.jts.Driver.connect(Driver.java:127)
weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.update(SourceFile:167)
com.bocom.cams.core.base.commons.BaseDao.update(BaseDao.java:37)
com.bocom.cams.core.base.commons.PersistableDao.update(PersistableDao.java:46)
com.bocom.cams.core.base.commons.PersistableService.modify(PersistableService.java:101)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:499)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
java.lang.Thread.run(Thread.java:662)
19:31:43.633 [hostName:scyy1] [pool-4514-thread-233] [2017080900000020] ERROR 报错信息如下：weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CAMSCORE is Suspended, cannot allocate resources to applications..
weblogic.jdbc.common.internal.JDBCUtil.wrapAndThrowResourceException(JDBCUtil.java:251)
weblogic.jdbc.pool.Driver.connect(Driver.java:142)
weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
weblogic.jdbc.jts.Driver.connect(Driver.java:127)
weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.insert(SourceFile:155)
com.bocom.cams.core.base.commons.BaseDao.save(BaseDao.java:26)
com.bocom.cams.core.base.commons.PersistableDao.insert(PersistableDao.java:36)
com.bocom.cams.core.base.commons.PersistableService.add(PersistableService.java:73)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:521)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
java.lang.Thread.run(Thread.java:662)
19:31:43.634 [hostName:scyy1] [pool-4514-thread-281] [2017080900000020] ERROR 报错信息如下：weblogic.jdbc.extensions.PoolDisabledSQLException: weblogic.common.resourcepool.ResourceDisabledException: Pool CAMSCORE is Suspended, cannot allocate resources to applications..
weblogic.jdbc.common.internal.JDBCUtil.wrapAndThrowResourceException(JDBCUtil.java:251)
weblogic.jdbc.pool.Driver.connect(Driver.java:142)
weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:658)
weblogic.jdbc.jts.Driver.connect(Driver.java:127)
weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:533)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:498)
weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:491)
org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
com.bocom.jump.bp.service.sqlmap.support.SqlSessionUtils.getSqlSession(SourceFile:116)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:56)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.a(SourceFile:67)
com.bocom.jump.bp.service.sqlmap.impl.SqlMapImpl.update(SourceFile:167)
com.bocom.cams.core.base.commons.BaseDao.update(BaseDao.java:37)
com.bocom.cams.core.base.commons.PersistableDao.update(PersistableDao.java:46)
com.bocom.cams.core.base.commons.PersistableService.modify(PersistableService.java:101)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService.interestCalculate(InterestCalculateService.java:499)
com.bocom.cams.core.cd.calcandclearinterest.service.InterestCalculateService$3.toProcess(InterestCalculateService.java:2241)
com.bocom.cams.core.base.commons.support.AsynchronousProcessor.run(AsynchronousProcessor.java:65)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
java.lang.Thread.run(Thread.java:662)

在 MOS 上输入关键字“Connection reset”，找到相关的文章：

Connection Reset When Creating a JDBC Pool Connection (文档 ID 1532894.1)

In this Document

Symptoms

Changes

Cause

Solution

References

Oracle WebLogic Server – Version 10.3.6 to 10.3.6
Information in this document applies to any platform.

Once a new connection pool is configured and testing is also successful, the managed server is restarted but it comes up in Admin State and shows JDBC error.

In the standard error, the following sequence of exceptions are shown:

<Feb 22, 2013 11:24:32 AM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING.>
<Feb 22, 2013 11:24:32 AM CST> <Info> <SAFService> <BEA-281003> <The SAF service has been initialized.>
<Feb 22, 2013 11:24:32 AM CST> <Info> <SAFService> <BEA-281002> <The SAF service has been started.>
<Feb 22, 2013 11:24:32 AM CST> <Info> <WseeCore> <BEA-220502> <The Wsee Service is starting>
<Feb 22, 2013 11:24:32 AM CST> <Info> <Deployer> <BEA-149209> <The Slave Deployer is resuming.>
<Feb 22, 2013 11:24:33 AM CST> <Info> <JDBC> <BEA-001177> <Creating data source connection pool named GTEM_DS, URL = jdbc:oracle:thin:@<URL>:<PORT>:<DB>, Properties = user=<USER>;.>
<Feb 22, 2013 11:24:35 AM CST> <Info> <Common> <BEA-000626> <Free resources in pool “GTEM_DS” will be tested every “120” seconds.>
<Feb 22, 2013 11:25:06 AM CST> <Info> <JDBC> <BEA-001508> <Destroying data source connection pool GTEM_DS.>
<Feb 22, 2013 11:25:06 AM CST> <Info> <JDBC> <BEA-001155> <The following exception has occurred:

weblogic.common.ResourceException: Unknown Data Source GTEM_DS
at weblogic.jdbc.common.internal.ConnectionPoolManager.shutdownConnectionPool(ConnectionPoolManager.java:630)
at weblogic.jdbc.common.internal.ConnectionPoolManager.shutdownAndDestroyPool(ConnectionPoolManager.java:602)
at weblogic.jdbc.common.internal.ConnectionPoolManager.shutdownAndDestroyPool(ConnectionPoolManager.java:533)
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:330)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:173)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:167)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.flow.ModuleStateDriver.prepare(ModuleStateDriver.java:38)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:139)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:57)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:707)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:237)
at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:158)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:66)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:40)
at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:22)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:166)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:187)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:95)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
>
<Feb 22, 2013 11:25:06 AM CST> <Error> <Deployer> <BEA-149205> <Failed to initialize the application “GTEM_DS” due to error weblogic.application.ModuleException:
weblogic.application.ModuleException:
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:337)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:173)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:167)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.flow.ModuleStateDriver.prepare(ModuleStateDriver.java:38)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:139)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:57)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:707)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:237)
at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:158)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:66)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:40)
at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:22)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:166)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:187)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:95)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: weblogic.common.ResourceException: weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Connection reset
at weblogic.jdbc.common.internal.ConnectionEnvFactory.createResource(ConnectionEnvFactory.java:288)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1310)
at weblogic.common.resourcepool.ResourcePoolImpl.makeResources(ResourcePoolImpl.java:1227)
at weblogic.common.resourcepool.ResourcePoolImpl.start(ResourcePoolImpl.java:250)
at weblogic.jdbc.common.internal.ConnectionPool.doStart(ConnectionPool.java:1397)
at weblogic.jdbc.common.internal.ConnectionPool.start(ConnectionPool.java:187)
at weblogic.jdbc.common.internal.ConnectionPoolManager.createAndStartPool(ConnectionPoolManager.java:521)
at weblogic.jdbc.common.internal.ConnectionPoolManager.createAndStartPool(ConnectionPoolManager.java:442)
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:288)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:173)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:167)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.flow.ModuleStateDriver.prepare(ModuleStateDriver.java:38)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:139)
at weblogic.application.internal.flow.DeploymentCallbackFlow.prepare(DeploymentCallbackFlow.java:57)
at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:707)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:35)
at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:237)
at weblogic.application.internal.SingleModuleDeployment.prepare(SingleModuleDeployment.java:48)
at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:158)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)
at weblogic.deploy.internal.targetserver.SystemResourceDeployment.prepare(SystemResourceDeployment.java:66)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:40)
at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:191)
at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:22)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:166)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:187)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:95)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:545)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
>

It is a brand new DS. Default DS configuration level was applied:

<?xml version=’1.0′ encoding=’UTF-8′?>
<jdbc-data-source xmlns=”http://xmlns.oracle.com/weblogic/jdbc-data-source” xmlns:sec=”http://xmlns.oracle.com/weblogic/security” xmlns:wls=”http://xmlns.oracle.com/weblogic/security/wls” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://xmlns.oracle.com/weblogic/jdbc-data-source http://xmlns.oracle.com/weblogic/jdbc-data-source/1.2/jdbc-data-source.xsd”>
<name>GTEM_DS</name>
<jdbc-driver-params>
<url>jdbc:oracle:thin:@[URL]:[PORT]:[DB]</url>
<driver-name>oracle.jdbc.OracleDriver</driver-name>
<properties>
<property>
<name>user</name>
<value>[USER]</value>
</property>
</properties>
<password-encrypted>[ENCLIPTED VALUE]</password-encrypted>
</jdbc-driver-params>
<jdbc-connection-pool-params>
<test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>
</jdbc-connection-pool-params>
<jdbc-data-source-params>
<jndi-name>GTEM_DS</jndi-name>
<global-transactions-protocol>OnePhaseCommit</global-transactions-protocol>
</jdbc-data-source-params>
</jdbc-data-source>

The issue is caused by the database not being able to respond to a large number of connections at the same time and hence throwing a connection reset.

Please follow these steps to resolve the issue: