Oracle Multimedia
这个特性在19c不再支持,参见说明。
不过多媒体文件仍建议存放在SecureFiles LOB中,参见此文。
SecureFiles初探
本实验参照Oracle Learning Library中的Using SecureFiles to Improve Performance, Maximize Storage, and Enhance Security。
关于SecureFiles,还有一篇很好的文章。
SecureFiles是BasicFiles的演进,11g开始支持。
SQL> show parameter db_securefile;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_securefile string PREFERRED
$ mkdir $ORACLE_HOME/wallet
编辑$ORACLE_HOME/network/admin/sqlnet.ora
文件,加入以下,以指定wallet位置:
ENCRYPTION_WALLET_LOCATION= (SOURCE=(METHOD=FILE)(METHOD_DATA= (DIRECTORY=/opt/oracle/product/19c/dbhome_1/wallet)))
解压sf.zip到临时目录,本例为/tmp。
进入临时目录,登录到数据库,运行脚本obe_setup.sql
,注意,由于我使用的是PDB,因此所有的connect语句均需修改,本例修改两处,为connect sys/Welcome1@orclpdb1 as sysdba
和connect sf_demo/oracle@orclpdb1
:
$ cd /tmp
$ ls -l
total 184
-rw-r--r--. 1 oracle oinstall 177 May 30 2007 check_encrypt.sql
-rw-r--r--. 1 oracle oinstall 37376 May 30 2007 david.sloan.doc
-rw-r--r--. 1 oracle oinstall 189 Mar 5 2007 dedup_compress.sql
-rw-r--r--. 1 oracle oinstall 177 May 30 2007 encrypt_lob.sql
-rw-r--r--. 1 oracle oinstall 64000 May 30 2007 karl.brimmer.doc
-rw-r--r--. 1 oracle oinstall 2554 May 30 2007 mig_bf_to_sf.sql
-rw-r--r--. 1 oracle oinstall 34304 May 30 2007 monica.petera.doc
-rw-r--r--. 1 oracle oinstall 8894 May 30 2007 obe_setup.sql
-rw-r--r--. 1 oracle oinstall 184 Apr 22 2009 read_data.sql
-rw-r--r--. 1 oracle oinstall 255 Mar 5 2007 reclaim_space.sql
-rw-r--r--. 1 oracle oinstall 118 May 30 2007 space_usage.sql
-rw-r--r--. 1 oracle oinstall 230 May 30 2007 write_data.sql
...
$ sqlplus /nolog
SQL> @obe_setup.sql
原脚本缺了给sf_demo用户赋表空间写的权限,补上:
SQL> alter user sf_demo quota unlimited on obe_tbs1;
User altered.
SQL> alter user sf_demo quota unlimited on obe_tbs2;
User altered.
另外,PDB中的wallet也需要打开,并创建master key,注意口令是oracle,与之前设定的一致:
SQL> show con_name;
CON_NAME
------------------------------
ORCLPDB1
SQL> ADMINISTER KEY MANAGEMENT SET KEYSTORE open identified by "oracle";
keystore altered.
SQL> administer key management create key identified by "oracle" with backup;
keystore altered.
SQL> administer key management use key 'AQQK0+b2fk/Pv7hk3BedhlEAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' identified by "oracle" with backup;
keystore altered.
SQL> select * from v$encryption_wallet;
WRL_TYPE WRL_PARA STATUS WALLET_TYPE WALLET_OR KEYSTORE FULLY_BAC CON_ID
-------------------- -------- -------- -------------------- --------- -------- --------- ----------
FILE OPEN PASSWORD SINGLE UNITED NO 3
然后,写数据,write_data.sql中的连接串也需改为PDB:
$ sqlplus /nolog
SQL> @write_data
Connected.
Begin inserting rows...
Row 1 inserted.
...
Row 30 inserted.
PL/SQL procedure successfully completed.
Begin inserting rows...
Row 1 inserted.
...
Row 30 inserted.
PL/SQL procedure successfully completed.
Begin inserting rows...
Row 1 inserted.
...
Row 30 inserted.
PL/SQL procedure successfully completed.
timing for: load_data
Elapsed: 00:00:00.30
总共90条记录,也就是3个doc文件,每个重复插入30次。
然后读取数据:
$ sqlplus /nolog
SQL> @read_data
SQL> @read_data
Connected.
SQL> set serveroutput on
SQL> set verify on
SQL> set term on
SQL> set lines 200
SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...
SQL>
SQL> timing start read_data
SQL> exec read_lob;
The length is: 64000
The ID is: 1
The blob is read: ??? > ?? x z ???? w
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
The length is: 64000
...
The length is: 64000
The ID is: 30
The blob is read: ??? > ?? x z ???? w
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
The length is: 34304
The ID is: 1
The blob is read: ??? > ?? > @ ???? =
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
...
The length is: 34304
The ID is: 30
The blob is read: ??? > ?? > @ ???? =
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
...
The length is: 37376
The ID is: 30
The blob is read: ??? > ?? D F ???? C
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
PL/SQL procedure successfully completed.
SQL> timing stop
timing for: read_data
Elapsed: 00:00:00.11
确认空间使用,输入BF,表示老的Basic File格式:
$ sqlplus /nolog
SQL> @space_usage
Connected.
SQL>
SQL> exec check_space_&Enter_BF_or_SF_for_LOB_type
Enter value for enter_bf_or_sf_for_lob_type: BF
FS1 Blocks = 0 Bytes = 0
FS2 Blocks = 0 Bytes = 0
FS3 Blocks = 0 Bytes = 0
FS4 Blocks = 0 Bytes = 0
Full Blocks = 540 Bytes = 4423680
Unformatted Blocks = 7618 Bytes = 62406656
=============================================
Total Blocks = 540 || Total Bytes = 4423680
PL/SQL procedure successfully completed.
将BasicFile LOB 迁移到 SecureFiles:
SQL> @mig_bf_to_sf
SQL> -- run this script as SYS
SQL> connect sys/Welcome1@orclpdb1 as sysdba
Connected.
SQL>
SQL> set echo on;
SQL> set serveroutput on
SQL> set term on;
SQL> set lines 200
SQL>
SQL> set pause off
SQL>
SQL> /*== Check the current segment type of the LOB ==*/
SQL> /*== ==*/
SQL> /*== Note that the segment subtype for ==*/
SQL> /*== BasicFile LOB storage is ASSM ==*/
SQL>
SQL> SELECT segment_name, segment_type, segment_subtype
2 FROM dba_segments
3 WHERE tablespace_name = 'OBE_TBS1'
4 AND segment_type = 'LOBSEGMENT'
5 /
SEGMENT_NAME SEGMENT_TYPE SEGMENT_SU
-------------------------------------------------------------------------------------------------------------------------------- ------------------ ----------
SYS_LOB0000073784C00004$$ LOBSEGMENT ASSM
SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...
SQL>
SQL> set pause on
SQL>
SQL> /*== Create the interim table for online redefinition ==*/
SQL> /*==
SQL> /*== Note that the recommended practice is to enable ==*/
SQL> /*== COMPRESSION and DEDUPLICATION at table createion ==*/
SQL> /*== time. Also, the ALTER TABLE SHRINK operation is ==*/
SQL> /*== not yet supported for SecureFiles LOBs in ==*/
SQL> /*== release 1 of Oracle Database 11g. ==*/
SQL>
SQL> CREATE TABLE sf_demo.resumes_interim
2 (id NUMBER, first_name VARCHAR2(15),
3 last_name VARCHAR2 (40), resume BLOB)
4 LOB (resume) STORE AS SECUREFILE
5 (TABLESPACE obe_tbs2
6 COMPRESS HIGH
7 DEDUPLICATE)
8 /
Table created.
SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...
SQL>
SQL> /*== Perform the online redefinition ==*/
SQL>
SQL> DECLARE
2 error_count PLS_INTEGER := 0;
3 BEGIN
4
5 DBMS_REDEFINITION.START_REDEF_TABLE
6 ('sf_demo', 'resumes', 'resumes_interim',
7 'id id, first_name first_name, last_name last_name, resume resume',
8 OPTIONS_FLAG => DBMS_REDEFINITION.CONS_USE_ROWID);
9
10 DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS
11 ('sf_demo', 'resumes', 'resumes_interim',
12 1, true,true,true,false, error_count);
13
14 DBMS_OUTPUT.PUT_LINE('Errors := ' || TO_CHAR(error_count));
15
16 DBMS_REDEFINITION.FINISH_REDEF_TABLE
17 ('sf_demo', 'resumes', 'resumes_interim');
18
19 END;
20 /
Errors := 0
PL/SQL procedure successfully completed.
SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...
SQL>
SQL> /*== Drop the interim table ==*/
SQL>
SQL> DROP TABLE sf_demo.resumes_interim
2 /
Table dropped.
SQL>
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...
SQL>
SQL> set pause off
SQL>
SQL> /*== Check the segment type of the migrated LOB ==*/
SQL> /*== ==*/
SQL> /*== Note that the segment subtype for ==*/
SQL> /*== SecureFiles LOB storage is SECUREFILE ==*/
SQL>
SQL> SELECT segment_name, segment_type, segment_subtype
2 FROM dba_segments
3 WHERE tablespace_name = 'OBE_TBS2'
4 AND segment_type = 'LOBSEGMENT'
5 /
SEGMENT_NAME SEGMENT_TYPE SEGMENT_SU
-------------------------------------------------------------------------------------------------------------------------------- ------------------ ----------
SYS_LOB0000073803C00004$$ LOBSEGMENT SECUREFILE
SQL> pause Press [Enter] to continue...
Press [Enter] to continue...
SQL>
SQL> set pause off
SQL>
SQL> /*== Check the DBA_LOBS data dictionary view for ==*/
SQL> /*== information on the SecureFiles COMPRESSION ==*/
SQL> /*== and DEDUPLICATION settings ==*/
SQL>
SQL> SELECT column_name, segment_name,
2 compression, deduplication, securefile
3 FROM dba_lobs
4 WHERE owner = 'SF_DEMO' and table_name = 'RESUMES'
5 /
COLUMN_NAME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SEGMENT_NAME COMPRE DEDUPLICATION SEC
-------------------------------------------------------------------------------------------------------------------------------- ------ --------------- ---
RESUME
SYS_LOB0000073803C00004$$ HIGH LOB YES
迁移这一步使用了online redefinition,这是EBR的前身,有点意思。
迁移后,虽然segment的类型均为LOBSEGMENT,但子类型由ASSM变为了SECUREFILE。压缩和重复数据删除都启用了。
查看空间使用,输入SF表示SecureFile:
SQL> @space_usage
SQL> connect sf_demo/oracle@orclpdb1
Connected.
SQL>
SQL> set serveroutput on
SQL> set pause off
SQL> set echo on
SQL>
SQL> exec check_space_&Enter_BF_or_SF_for_LOB_type
Enter value for enter_bf_or_sf_for_lob_type: SF
Segment Blocks = 8192 Bytes = 67108864
Used Blocks = 5 Bytes = 40960
Expired Blocks = 7962 Bytes = 65224704
Unexpired Blocks = 145 Bytes = 1187840
=============================================
PL/SQL procedure successfully completed.
注意到Used Block为5,而迁移前为540。
性能方面,SecureFile同样有优势,此处略。
最后是加密部分:
SQL> @encrypt_lob
SQL>
SQL> connect sys/Welcome1@orclpdb1 as sysdba
Connected.
SQL>
SQL> ALTER TABLE sf_demo.resumes
2 MODIFY (resume ENCRYPT USING 'AES192')
3 /
Table altered.
加密这部分,参考了此文档。
确认已加密:
SQL> @check_encrypt
SQL> connect sf_demo/oracle@orclpdb1
Connected.
SQL> set pause off
SQL> set echo on
SQL>
SQL> col TABLE_NAME format a18;
SQL> col COLUMN_NAME format a19;
SQL> col ENCRYPTION_ALG format a17;
SQL>
SQL> SELECT *
2 FROM USER_ENCRYPTED_COLUMNS
3 /
TABLE_NAME COLUMN_NAME ENCRYPTION_ALG SAL INTEGRITY_AL
------------------ ------------------- ----------------- --- ------------
RESUMES RESUME AES 192 bits key YES SHA-1
DBFS初探
本实验参照ORACLE-BASE上的文章:Oracle Database File System (DBFS) in Oracle Database 11g Release 2。
文章很全面,但我的环境是19.3,使用的是PDB,和它的11gR2有些区别。
创建表空间:
connect sys/Welcome1@orclpdb1 as sysdba
CREATE TABLESPACE dbfs_ts
DATAFILE '/u01/app/oracle/oradata/DB11G/dbfs01.dbf'
SIZE 1M AUTOEXTEND ON NEXT 1M;
创建文件系统:
cd $ORACLE_HOME/rdbms/admin
sqlplus dbfs_user/dbfs_user@orclpdb1
SQL> @dbfs_create_filesystem.sql dbfs_ts staging_area
安装FUSE,linux下的DBFS需要FUSE(Filesystem in Userspace):
# yum install kernel-devel fuse fuse-libs
Mount文件系统:
# mkdir /mnt/dbfs
# chown oracle:oinstall /mnt/dbfs
配置动态链接库路径,参照文章12cR2部分,根据实际环境修改ORACLE_HOM环境变量:
# echo "/usr/local/lib" >> /etc/ld.so.conf.d/usr_local_lib.conf
# export ORACLE_HOME=/opt/oracle/product/19c/dbhome_1
# ln -s $ORACLE_HOME/lib/libclntsh.so.12.1 /usr/local/lib/libclntsh.so.12.1
# ln -s $ORACLE_HOME/lib/libnnz12.so /usr/local/lib/libnnz12.so
# ln -s /lib64/libfuse.so.2 /usr/local/lib/libfuse.so.2
# ln -s /lib64/libfuse.so.2 /usr/local/lib/libfuse.so
# ldconfig
修改/etc/fuse.conf
文件,解除user_allow_other
的注释:
# mount_max = 1000
user_allow_other
修改fusermount
的权限:
# chmod +x /usr/bin/fusermount
修改/etc/abrt/abrt-action-save-package-data.conf
部分,因我Linux 7环境下无此目录,略过。
重启服务器:
# shutdown -r now
在oracle用户的.bash_profile文件中添加以下:
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib
mount文件系统:
$ dbfs_client dbfs_user@orclpdb1 /mnt/dbfs
Password: dbfs_user
这是个前台命令,通过df可确认文件系统已mount:
$ df /mnt/dbfs
Filesystem 1K-blocks Used Available Use% Mounted on
dbfs-dbfs_user@orclpdb1:/ 960 128 832 14% /mnt/dbfs
只有oracle用户可看到,root用户不行。
可以看到之前定义的staging_area
文件系统已mount:
$ ls -al /mnt/dbfs/staging_area
total 0
drwxrwxrwx. 4 root root 0 Dec 20 17:34 .
drwxr-xr-x. 3 root root 0 Dec 21 12:21 ..
drwxr-xr-x. 7 root root 0 Dec 20 08:13 .sfs
root用户可umount文件系统:
# fusermount -u /mnt/dbfs
dbfs_client命令示例:
# 查询文件系统
$ dbfs_client dbfs_user@orclpdb1 --command ls -a -l dbfs:/staging_area/
Password:
drwxr-xr-x root root 0 Dec 20 08:13 dbfs:/staging_area/.sfs
# 创建目录
$ dbfs_client dbfs_user@orclpdb1 --command mkdir dbfs:/staging_area/test_dir
Password:
# 查询确认
$ dbfs_client dbfs_user@orclpdb1 --command ls -a -l dbfs:/staging_area/
Password:
drwxr-xr-x oracle oinstall 0 Dec 21 13:10 dbfs:/staging_area/test_dir
drwxr-xr-x root root 0 Dec 20 08:13 dbfs:/staging_area/.sfs
# 拷贝文件
$ dbfs_client dbfs_user@orclpdb1 --command cp /tmp/flatxn.zip dbfs:/staging_area/test_dir/
Password:
/tmp/flatxn.zip -> dbfs:/staging_area/test_dir/flatxn.zip
# 删除文件
$ dbfs_client dbfs_user@orclpdb1 --command rm dbfs:/staging_area/test_dir/flatxn.zip
Password:
unlinking file dbfs:/staging_area/test_dir/flatxn.zip
# 再次拷贝
$ dbfs_client dbfs_user@orclpdb1 --command cp /tmp/flatxn.zip dbfs:/staging_area/test_dir/
Password:
/tmp/flatxn.zip -> dbfs:/staging_area/test_dir/flatxn.zip
DBFS还可以作为一种共享机制,只需在客户端安装dbfs_client即可,参见文章。
问题是,使用dbfs_client操作文件和使用操作系统下的命令操作有何区别?实际上,使用操作系统命令在服务器端操作是可以的,因为DBFS符合POSIX标准,但在客户端,你可以使用dbfs_client或scp。
$ cp /vagrant/dragonball.jpg /mnt/dbfs/staging_area/test_dir/
通过user_lobs,可知DBFS存放元数据的表为SFS$_FST_1。查询此表:
对于DBFS,可以使用PL/SQL API及系统视图得到许多信息,以下命令来自此文:
-- DBMS_DBFS_CONTENT_SFS
SELECT * FROM TABLE(dbms_dbfs_sfs.listTables);
SELECT * FROM TABLE(dbms_dbfs_sfs.listFilesystems);
SELECT * FROM TABLE(dbms_dbfs_sfs.listVolumes);
SELECT * FROM TABLE(dbms_dbfs_sfs.listSnapshots);
-- DBMS_DBFS_CONTENT
SELECT * FROM TABLE(dbms_dbfs_content.liststores);
SELECT * FROM TABLE(dbms_dbfs_content.listmounts);
SELECT * FROM TABLE(dbms_dbfs_content.listallcontent);
SELECT * FROM TABLE(dbms_dbfs_content.listallproperties);
SELECT * FROM TABLE(dbms_dbfs_content.list(path => '/', recurse => 1, store_name => 'staging_area'));
-- Views
-- 除此命令外,其它命令均测试成功
SELECT UTL_RAW.cast_to_varchar2(filedata) AS filedata
FROM dbfs_content
WHERE pathtype = 'file';
SELECT * FROM dbfs_content_properties;
实际文件物理存放在表dbfs_content中:
最后,单个LOB最大为8TB,参见说明。