Monday, October 14, 2013

How to monitor db2fmp ?

How to monitor db2fmp ?

Technote (FAQ)

Question
How does the db2fmp process relate to an application? How do I determine which application is calling a routine executing inside this db2fmp process?
What is the criteria to reuse db2fmp process?
Answer
From a DB2 perspective we can determine the application and routine calling the routine along with some diagnostics to help the developer that wrote the routine troubleshoot any performance problems.
What is the criteria to reuse db2fmp process ?

Threadsafe routines: One db2fmp can run many routines servicing multiple application handles (connections).

Non-threadsafe routines: One db2fmp can run ONE routine servicing ONE application handle.

By default DB2 pools inactive 32-bit (db2fmp32) and 64-bit (db2fmp) processes up to the value specified by FENCED_POOL. Checking the UNIX/Linux "ps" output to look for "db2fmp (idle)" processes indicates there is no db2fmp associated with a DB2 co-ordinator agent, this has no bearing regarding whether it is active or inactive.

FENCED_POOL determines the number of pooled threads inside one db2fmp process if it is running threadsafe routines.  For non-threaded routines this parameter controls the total number of db2fmp processes.  In our case db2fmp32 always runs as non-threaded.  For example If we have FENCED_POOL=10, we can have 10 inactive non-threaded db2fmp processes plus 5 inactive threaded db2fmp processes for a total of 15 inactive db2fmp processes.

If Connection A called routine SPTEST, disconnects and calls the SPTEST is the same db2fmp process reused? No. Once Connection A terminates the db2fmp is returned to the pool. When it reconnects DB2 will start a new db2fmp process.

How do we know if the db2fmp process is active/inactive ?

The active/inactive state can be monitored via db2pd -fmp which will provide an entry for each db2fmp process along with all threads inside. In some cases you may see multiple active threads because each thread is executing a different routine (stored procedure/user defined function) on behalf of multiple applications.

The output below indicates there is no db2fmp associated with a DB2 agent. This has no bearing regarding the active/inactive state of the db2fmp.

db2fmp (idle)

The output below indicates the db2fmp process is a single threaded routine which has been associated with DB2 agent thread #1234

db2fmp32 (1234)
db2fmp64 (1234)

How do I determine which application is calling a routine executing inside this db2fmp process?

To determine if a routine is threadsafe check the CREATE PROCEDURE DDL statement.

In this example you can see that PID 6422648 is working on behalf of DB2's coordinator agent 77811. An application snapshot will provide a coordinator PID. Once you have the application handle the application snapshot should be able to provide more information such as dynamic SQL statement, or package in the case of a stored procedure. There are alternatives to using get snapshot such as using the db2pd -applications -alldbs.

[

db2 "get snapshot for all applications"

Application Snapshot

Application handle = 1283
Application status = Connect Completed
Status change time = Not Collected
Application code page = 1208
...
Coordinator agent process or thread ID = 77811

db2pd -fmp

FMP:
Pool Size: 99
Max Pool Size: 100
Keep FMP: YES
Initialized: YES
Trusted Path: /home/db2inst1/sqllib/function/unfenced
Fenced User: db2fenc1
Shared Memory: 0x0780000000790420
IPC Pool: 0x0780000000790480

FMP Process:
Address FmpPid Bit Flags ActiveThrd PooledThrd ForcedThrd Active IPCList
0x0780000001D3EE60 6422648 64 0x00000000 0 0 0 Yes 0x0780000001DD4E20

Active Threads:
Address FmpPid EduPid ThreadId
0x0780000001D3EE78 6422648 77811 0

Pooled Threads: (these are inactive threads)
Address FmpPid ThreadId
No pooled threads.

Forced Threads:
Address FmpPid ThreadId
No forced threads.

Once a routine is finished executing it can be used by a completely different application. The command below will help you identify which routines were executed in the past. This feature was introduced in v9.5 FP5 and v9.7 FP1.

We can see that the stored procedure last executed was SYSPROC.SYSINSTALLOBJECTS. Normally you would see a C=C or J=Java but P probably means its a built in DB2 routine.

$ db2pd -fmpexechistory

Database Partition 0 -- Active -- Up 2 days 13:28:24 -- Date 10/04/2011 11:49:05

FMP Process:
FmpPid Bit Flags ActiveThrd PooledThrd ForcedThrd Active
20709658 64 0x00000002 0 1 0 Yes

Active Threads:
No active threads.

Pooled Threads:
ThreadId : 772
Routine ID Timestamp
65818 2011-10-03-16.35.49.678540

Forced Threads:
No forced threads.

$ db2 "select routineschema, routinename, routinetype from sysibm.sysroutines where routine_id=65818"

ROUTINESCHEMA ROUTINENAME ROUTINETYPE
-------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- -----------
SYSPROC SYSINSTALLOBJECTS P

1 record(s) selected

Monitoring Memory/CPU

The principle goal of running third party applications inside a separate process which we decide to call db2fmp is to isolate untrusted code from the DB2 engine to reduce the possibility of bringing down the DB2 instance. Essentially all DB2 does is fork() a new process and setup a communications buffer to the process via a shared memory segment and load the libraries needed to run the routine.

The memory and CPU is managed by the JVM if running a Java routine and C if running a C routine. Since troubleshooting a db2fmp process is similar to troubleshooting a third party application, we have to use the tools provided at the operating system level.

Please see link below "Collecting data: DB2 Stored Procedure Problems" Non-SQL section for diagnostics to collect to help the application developer that wrote the non-SQL routine troubleshoot the root cause of the problem. DB2 Support may also request this information to help guide your application developer as well.

Friday, May 10, 2013

SQL0805N 找不到程序包 "NULLID.SQLUFG13 0X4141414141594758"。SQLSTATE=51002 解决


开发使用Quest Central 进行 import 操作,语句如下:

import from "d:\A_MOBILEAREA.txt" of del insert into A_MOBILEAREA(MOBILE,AREACODE)

操作报错:

SQL0805N 找不到程序包 "NULLID.SQLUFG13 0X4141414141594758"SQLSTATE=51002

查询帮助文档:

SQL0805N  Package "<package-name>" was not found.

………省略…………..

User response:

Specify the correct package name or bind the program. If the application
being run is not bound to the database, contact the database
administrator to do the necessary binding. Ensure that the application
or object module being executed is the compiled and linked modified
source code associated with the precompile and bind that generated the
package.

If the CURRENT PACKAGE PATH is set, ensure that the schema that includes
the package is specified in the CURRENT PACKAGE PATH.

The following SQL statements can be used to query the catalog to
determine if there are different versions of the package.

SELECT PKGSCHEMA, PKGNAME,
       PKGVERSION, UNIQUE_ID
FROM SYSCAT.PACKAGES
WHERE PKGSCHEMA = 'pkgschema'
      and PKGNAME='pkgname'.

Note that the UNIQUE_ID column corresponds to the consistency token.

If the DB2 utility programs need to be rebound to the database, the
database administrator can accomplish this by issuing one of the
following CLP command from the bnd subdirectory of the instance, while
connected to the database:

*  "db2 bind @db2ubind.lst blocking all grant public" for the DB2
   utilities.
*  "db2 bind @db2cli.lst blocking all grant public" for CLI.

Federated system users: ensure that the packages required for the
federated server are bound at the applicable data sources. Refer to the
Federated Systems Guide for more information about binding packages to
data sources.

sqlcode: -805

sqlstate: 51002

SQL0805N帮助已经告知如果是utilityimport操作属于utility 需要重新绑定可以执行 db2 bind @db2ubind.lst blocking all grant public

于是乎登录到服务器上执行sqllib/bnd下执行bind命令,执行结果虽然有些警告但是执行正常,没有任何错误。详见:

[db2inst1@localhost94:/home/db2inst1]$ db2 bind /home/db2inst1/sqllib/bnd/@db2ubind.lst blocking all

LINE    MESSAGES FOR db2ubind.lst
------  --------------------------------------------------------------------
        SQL0061W  The binder is in progress.

LINE    MESSAGES FOR db2clpnc.bnd
------  --------------------------------------------------------------------
        SQL0595W  Isolation level "NC" has been escalated to "UR". 
                  SQLSTATE=01526

LINE    MESSAGES FOR db2arxnc.bnd
------  --------------------------------------------------------------------
        SQL0595W  Isolation level "NC" has been escalated to "UR". 
                  SQLSTATE=01526

LINE    MESSAGES FOR db2ats_sps.bnd
------  --------------------------------------------------------------------
 1173   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1203   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1234   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1482   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1499   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1517   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1555   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1679   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1696   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1715   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1732   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name.  
                  SQLSTATE=01532
 1895   SQL0204N  "SYSTOOLS.ADMINTASKSTATUS" is an undefined name. 
                  SQLSTATE=01532
 1950   SQL0204N  "SYSTOOLS.ADMINTASKSTATUS" is an undefined name. 
                  SQLSTATE=01532
 1962   SQL0204N  "SYSTOOLS.ADMINTASKS" is an undefined name. 
                  SQLSTATE=01532
 1979   SQL0204N  "SYSTOOLS.ADMINTASKSTATUS" is an undefined name. 
                  SQLSTATE=01532

LINE    MESSAGES FOR db2_adminotm.bnd
------  --------------------------------------------------------------------
  342   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  371   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  455   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  536   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  570   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  590   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  671   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  767   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532
  903   SQL0204N  "SYSTOOLS.ADMIN_MOVE_TABLE" is an undefined name. 
                  SQLSTATE=01532

LINE    MESSAGES FOR db2ubind.lst
------  --------------------------------------------------------------------
        SQL0091N  Binding was ended with "0" errors and "26"
                  warnings.

这两个警告可以忽略,因为我们目前用不到。
执行完bind后依然没有找到SQLUFG13包,命令:

db2 "SELECT PKGSCHEMA, PKGNAME, PKGVERSION, UNIQUE_ID FROM SYSCAT.PACKAGES WHERE PKGNAME = 'SQLUFG13'"

反复检查所做过的操作并没有什么错误。于是上网查询得到的解决方法:

解决步骤:
 开始-》运行: db2cmd

1db2 connect to sample user XXX using password 

2
db2 bind 'C:\Program Files\IBM\SQLLIB\bnd\@db2ubind.lst' blocking all

3
db2 bind 'C:\Program Files\IBM\SQLLIB\bnd\@db2cli.lst' blocking all


说明:

1XXX 为数据库用户名,password 为密码 
2C:\Program Files\IBM\SQLLIB    db2按照目录,换成自己的


初看网上找到的解决办法和我的操作时一样的,差别是我的环境是aix而它是windows环境。在反复核思考和核对操作时突然意识到开发使用的Quest Central不也是在windows上嘛,难道SQLUFG13包只在windows环境中有?

改变操作方式,通过Quest Central的所在windows上使用db2cmd连接到服务器上的数据库,然后在本地的db2安装目录上找到bnd目录执行bind,执行完后再次查看服务器的包列表终于出现SQLUFG13了。再次执行import操作正常完成!

解决这个问题后让我又再次想到一句经典的话:网络上的坑真的很多啊。
另外想抱怨的是:IBM,你把信息中心做得那么好为什么不介绍下系统自带的那些包呢?因为IBM的包还是很特别的,在其他数据库上并没有这样的东东,然后这方面的资料又很少。

Thursday, April 25, 2013

CentOS SSH配置


默认CentOS已经安装了OpenSSH,即使你是最小化安装也是如此。所以这里就不介绍OpenSSH的安装了。

SSH配置:
1、修改vi /etc/ssh/sshd_config,根据模板将要修改的参数注释去掉并修改参数值:
Port 22 指定SSH连接的端口号,安全方面不建议使用默认22端口
Protocol 2,1 允许SSH1和SSH2连接,建议设置成 Protocal 2
其他参数根据自己的需要进行调整。配置方法详见: man ssh_config
2、修改hosts.deny 在最后面添加一行:
sshd:All
3、修改hosts.allow 在最后面添加一行:
sshd:All
如果为了安装可以限制访问的IP,设置如下:
sshd:192.168.0.101
sshd:192.168.0.102
上述配置表示只允许101和102的服务器进行SSH连接
4、启动SSH
/etc/init.d/sshd start

至此SSH已经可以连接了
输入连接IP
配置相关参数
选择utf-8支持中文显示
自动输入登录用户root
输入用户名后就可以连接到服务器

但是目前我们的SSH连接还需要输入密码,下面将介绍使用密钥进行连接,免去了输入密码的烦恼:
1、在被管理机上生产密钥


复制代码
 1 [root@localhost ~]# mkdir /root/.ssh
 2 [root@localhost ~]# ssh-keygen -t rsa
 3 Generating public/private rsa key pair.
 4 Enter file in which to save the key (/root/.ssh/id_rsa):   ##直接回车默认路径
 5 Enter passphrase (empty for no passphrase):                ##输入密码短语
 6 Enter same passphrase again:                               ##重复密码短语
 7 Your identification has been saved in /root/.ssh/id_rsa.   ##如果在这里报错是因为SeLinux引起的按照Policy:yum install selinux-policy
 8 Your public key has been saved in /root/.ssh/id_rsa.pub.
 9 The key fingerprint is:
10 aa:76:71:1e:51:fe:3b:4c:51:30:b2:90:55:e9:58:7c root@localhost.localdomain
11 The key's randomart image is:
12 +--[ RSA 2048]----+
13 |        .ooo+o   |
14 |        ...o+.E  |
15 |         o.+ o   |
16 |        . o o    |
17 |        S. . .   |
18 |      ..o   o    |
19 |      .+ . o .   |
20 |    ... .   +    |
21 |   ...       .   |
22 +-----------------+
复制代码
2、putty生产密钥
打开puttygen,如果没有该程序可以到putty官方网站下载。
点击Generate按钮后出现下图,在红框中不断移动鼠标知道密钥生成完成

上面的大红框就是我们生成的公钥、这个公钥用于放在被管理服务器上,而私钥放在自己的机器上。
Key comment是一个备注信息,如果是企业环境那么会有很多的公钥在一台机器上,为了识别的话一般都会根据每个人的自己定义一个备注。
可以使用邮箱或者工号,输入Key comment先复制下公钥,并点击Saved public Key 和 Saved Private Key保存两个密钥
接着打开密钥代理工具pageant.exe(同样可以在putty的官网下载),使用pageant.exe有一个很大的好处,如果你还需要使用putty的其他工具那么他们可以共享密钥验证,而不需要反复去设置密钥。
任务栏的pageant.exe图标右键选择view keys打开下面窗口。
点击Add key添加刚才保存的私钥。

3、被管理机密钥部署
将被管理机上刚才生产的id_rsa.pub复制成authrized.keys
[root@localhost .ssh]# cp id_rsa.pub authrized.keys
[root@localhost .ssh]# chmod 600 authrized.keys      ##这一步是必须的,否则连接不上
修改vi /root/.ssh/authrized.keys 删除原来的密钥,添加puttygen.exe生产的密钥(也就是前面复制的公钥)
复制代码
sh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAh+gDpVSNIwBHZvmHS240AoueNHIgDHhczQ/fhiN/IdAQVdh7Ovw2pnJ4sd6so0kqCizsU7FOu2rvaK7vHC3QrrYmeqn94V595pYGLnMCbtEd7ONew47TU8wjtdldbc7liEmkTVIdkCbbrzQa372/u2LSjkldu2BUiXkevlnGNUc= hellwen.wu
~
~
~
~
复制代码
保存退出。
4、打开putty登录
注意看上图中红色部分,如果你是经常连接这台机器的话建议输入IP并配置要后再Saved Sessions里输入识别名称并点击保存按钮,这样在列表框中就会长期保留该服务器的连接配置。



Wednesday, April 24, 2013

Centos 6.2 安装 Pythonbrew

安装pythontools
# yum install gcc patch zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel
# yum groupinstall "Development tools"
# yum install python-setuptools
安装pythonbrew
# easy_install pythonbrew
初始化pythonbrew环境
# su - [username]
$ pythonbrew_install
根据提示修改.bashrc
$ echo "source ~/.pythonbrew/etc/bashrc" >> ~/.bashrc

列出可安装的Python版本
$ pythonbrew list --know
安装python
$ pythonbrew install 2.7.3
创建虚拟环境

$ pythonbrew venv create [env_name] -p 2.7.3
使用创建的虚拟环境
$ pythonbrew venv use [env_name]













Thursday, April 11, 2013

db2 联邦数据库练习


当我们需要对同一实例上两个数据库进行连接查询的时候是无法直接进行的。这一点在SQLServer中做得很好,可以直接使用数据库名称引用。而db2中必须使用联邦数据库(FEDERATED)
同一服务器上的两个实例:db2inst1 db2inst2 数据库分别是:mydb1 mydb2 (本地联邦和远程联邦同理,所以这里只演示远程联邦)
下面的操作在db2inst1上对db2inst2进行联邦,也就是可以在db2inst1上通过nickname访问db2inst2,所有操作都是在db2inst1下进行:

db2 update dbm cfg using FEDERATED YES
db2 connect to mydb1
--这里需要注意的是这个语句不能使用db2 ""执行,而是需要进入到db2 Command Line Processor 交互模式。这点是进行了很多尝试才发现的问题,注意问题是在password后面的字符串必须是双引号
--drda是默认的适配器
db2
create server f_mydb2 type db2/udb version 9.7 wrapper drda authorization db2inst2 password "******" options(dbname 'mydb2')
db2 "create user mapping for "db2inst1" server f_mydb2 options (add remote_authid 'db2inst2' , add remote_password '******')"
--db2inst2中存在一个表tb2
db2 "create nickname mydb2_tb2 for f_mydb2.db2inst2.tb2"
db2 "select * from mydb2_tb2"
db2 "select * from tb1 a inner join mydb2_tb2 b on b.id = a.id"

上面配置中需要注意的:

  1. 如果没有适配器可以使用Craete Wrapper创建一个适配器,如:Create wrapper drda 采用默认lib创建db2的适配器
  2. 因为语法原因,create server语句可能会需要在db2下执行,而不能以db2 ""方式执行
  3. create nickname时一定要记得必须在user mapping的用户下创建,否则无效



另外附上相关的表和视图:
SYSIBM.SYSWRAPPERS --每一行代表注册过的转换器 (wrapper) 
SYSIBM.SYSSERVERS  --每一行代表了数据源的信息,如数据源版本号,类型等
SYSIBM.SYSFUNCMAPPINGS   --包括函数映射的基本信息如函数映射名,基于服务器的名字等等,一般可以用来判断找不到相应函数这类的错误
SYSIBM.SYSSERVEROPTIONS   --包括对服务器选项的设置信息,如 Collating_sequence,PROXY_AUTHID 等,这些选项的设置可以对查询性能产生影响
SYSIBM.SYSTYPEMAPPINGS   --包括数据类型映射的基本信息,如联邦服务器这一侧的数据类型和远程数据源短的数据类型的对应关系,一般用于判断数据类型相关的错误
SYSIBM.SYSUSEROPTIONS   --在这个表中包括了用户选项及其设置值,如连接远程数据源使用的用户名和密码信息 (REMOTE_AUTHID, REMOTE_PASSWORD) 等,一般可以用于判断连接错误或者访问权限错误 ( 配合 DB2 本身的 SYSDBAUTH 系统表 ) 
SYSIBM.SYSWRAPOPTIONS   --其中包含了关于创建的转换器的一些可选信息,例如在使用转换器的时候是否使用“ Trusted ”或者“ Fenced ”选项
SYSIBM.SYSPASSTHRUAUTH
SYSIBM.SYSFUNCMAPPARMOPTIONS
SYSIBM.SYSFUNCMAPOPTIONS

视图:
SYSCAT.FUNCMAPPINGS 
SYSCAT.FUNCMAPPARMOPTIONS 
SYSCAT.FUNCMAPOPTIONS 
SYSCAT.NICKNAMES 
SYSCAT.PASSTHRUAUTH 
SYSCAT.SERVEROPTIONS 
SYSCAT.SERVERS 
SYSCAT.TYPEMAPPINGS 
SYSCAT.USEROPTIONS 
SYSCAT.ROUTINESFEDERATED