加载中…
个人资料
  • 博客等级:
  • 博客积分:
  • 博客访问:
  • 关注人气:
  • 获赠金笔:0支
  • 赠出金笔:0支
  • 荣誉徽章:
正文 字体大小:

postgreSQL Ran out of memory retrieving query results错误

(2012-08-07 20:42:07)
标签:

杂谈

分类: jdbc编程

当postgresQL中表的数据量很大,而我们又想读取全部数据时,该怎么处理呢?如果一次直接获取

ResultSet rs = s.executeQuery("select * from " + tabName);会在查询时报“org.postgresql.util.PSQLException: Ran out of memory retrieving query results.”的错误

解决方法:

1.当获取大数据集时使用limit 和 offset 分多次获取,但这样要多次查询数据库,而且我感觉多次读取不能保证读取的是全集,即使使用排序,也不敢保证。

2.使用jdbc 驱动来处理:(1)连接处于非自动提交状态  (2)statement 被创建为 ResultSet.TYPE_FORWARD_ONLY.模式 (3) 使用setFetchSize(n); 避免读取全部结果集到内存。

postgreSQL文档如下:

Getting results based on a cursor

By default the driver collects all the results for the query at once. This can be inconvenient for large data sets so the JDBC driver provides a means of basing a ResultSet on a database cursor and only fetching a small number of rows.

A small number of rows are cached on the client side of the connection and when exhausted the next block of rows is retrieved by repositioning the cursor.

Note:

  • Cursor based ResultSets cannot be used in all situations. There a number of restrictions which will make the driver silently fall back to fetching the whole ResultSet at once.

  • The connection to the server must be using the V3 protocol. This is the default for (and is only supported by) server versions 7.4 and later.-

  • The Connection must not be in autocommit mode. The backend closes cursors at the end of transactions, so in autocommit mode the backend will have closed the cursor before anything can be fetched from it.-

  • The Statement must be created with a ResultSet type of ResultSet.TYPE_FORWARD_ONLY. This is the default, so no code will need to be rewritten to take advantage of this, but it also means that you cannot scroll backwards or otherwise jump around in the ResultSet.-

  • The query given must be a single statement, not multiple statements strung together with semicolons.

Example 5.2. Setting fetch size to turn cursors on and off.

Changing code to cursor mode is as simple as setting the fetch size of the Statement to the appropriate size. Setting the fetch size back to 0 will cause all rows to be cached (the default behaviour).

// make sure autocommit is off 
conn
.setAutoCommit(false);
 
Statement st = conn.createStatement();
 
 
// Turn use of the cursor on.
 
st
.setFetchSize(50);
 
ResultSet rs = st.executeQuery("SELECT * FROM mytable");
 
while (rs.next()) {
 
   
System.out.print("a row was returned.");
 
}
 
rs
.close();
 
 
// Turn the cursor off.
 
st
.setFetchSize(0);
 
rs
= st.executeQuery("SELECT * FROM mytable");
 
while (rs.next()) {
 
   
System.out.print("many rows were returned.");
 
}
 
rs
.close();
 
 
// Close the statement.
 
st
.close();
 

 

 

 


0

阅读 收藏 喜欢 打印举报/Report
  

新浪BLOG意见反馈留言板 欢迎批评指正

新浪简介 | About Sina | 广告服务 | 联系我们 | 招聘信息 | 网站律师 | SINA English | 产品答疑

新浪公司 版权所有