Who's online

There are currently 0 users and 39 guests online.

Recent comments


Oakies Blog Aggregator

A look into the Exadata infrastructure

An Oracle Exadata database machine consists of several parts: Intel based servers, infiniband switches, a cisco ethernet switch, a KVM switch and the hardware surrounding it like cables, 19″ rack, power distribution units. The Intel based servers are what “Exadata administrators” are administering the most. The intention of this article is to let the reader gain a little more insight into Exadata specific administration on those.

Two server layers: computing and storage
The two layers have quite different properties: the Exadata computing layer is Linux with Oracle grid infrastructure and the Oracle database software installed, very much as you would do yourself (if you install it in a strict OFA way), and the storage layer is Linux too, but with specific Exadata storage software.

Computing layer
The computing layer filesystems are using LVM on newer (X2) versions of Exadata, and using plain partitions without LVM on the V2 version of Exadata. There are 3 filesystems on a X2 computing node:

[root@xxxxdb01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
30G 14G 15G 49% /
/dev/sda1 124M 16M 102M 14% /boot
99G 29G 65G 32% /u01
tmpfs 81G 196M 81G 1% /dev/shm

These are two filesystems inside logical volume, and the boot partition on a plain partition. The /boot partition consists of the grub configuration, kernel and initrd (initial ramdisk). The root logical volume (LVDbSys1) contains the linux operating system files, and the /u01 logical volume (LVDbOra1) contains the Oracle grid/cluster and database software.

A nice thing to know about the computing layer storage is that there is space left in the single volume group which is used for the logical volumes:

[root@emc1db01 ~]# vgdisplay
--- Volume group ---
VG Name VGExaDb
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
Cur LV 3
Open LV 3
Max PV 0
Cur PV 1
Act PV 1
VG Size 557.62 GB
PE Size 4.00 MB
Total PE 142751
Alloc PE / Size 39424 / 154.00 GB
Free PE / Size 103327 / 403.62 GB <<-- 400GB space left!
VG UUID AZbMbf-papI-6FE2-hYlR-bzE1-G4pp-Bou81e

The disk configuration on the computing layer is build upon a RAID 5 set of 3 disks with a hot spare. The RAID is hardware based (the diskcontroller is reponsible for the RAID maintenance). This is visible through the MegaCli64 command:

[root@emc1db01 ~]# /opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -LALL -aALL

Adapter 0 -- Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name : DBSYS
RAID Level : Primary-5, Secondary-0, RAID Level Qualifier-3
Size : 557.75 GB
State : Optimal
Stripe Size : 1.0 MB
Number Of Drives : 3
Span Depth : 1
Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Access Policy : Read/Write
Disk Cache Policy : Disabled
Encryption Type : None

Number of Dedicated Hot Spares: 1
0 : EnclId - 252 SlotId - 3

Exit Code: 0x00

Let me emphasize this disks/RAID-set only is responsible for serving the operating system and accompanying files, and the grid/clusterware and database executables. All true database processing is done on the storage layer nodes.

Storage layer
The storage layer filesystems are not using LVM, nor regular partitions (/dev/sd* for disks visible as SCSI devices for linux):

[root@xxxxcel01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md5 9.9G 3.4G 6.0G 37% /
tmpfs 12G 0 12G 0% /dev/shm
/dev/md7 2.0G 623M 1.3G 33% /opt/oracle
/dev/md4 116M 37M 74M 34% /boot
/dev/md11 2.3G 222M 2.0G 10% /var/log/oracle

The devices used are linux software RAID devices (‘md’ means multiple devices, which is a linux device which is build from independed underlying devices). The reason for using linux software RAID for the operating system and Oracle storage software is to provide redundancy, because the underlying disks on the storage servers do not provide that. Redundancy/error protection for the databases data on the storage servers is done using ASM normal redundancy, which is, put blunt and simple, RAID 10 handled on the ASM layer.

A logical next question is: okay, how do these linux software RAID/MD devices look like? Well, the most simple way is to use /proc/mdstat:

[root@emc1cel01 ~]# cat /proc/mdstat
Personalities : [raid1]
md4 : active raid1 sdb1[1] sda1[0]
120384 blocks [2/2] [UU]

md5 : active raid1 sdb5[1] sda5[0]
10482304 blocks [2/2] [UU]

md6 : active raid1 sdb6[1] sda6[0]
10482304 blocks [2/2] [UU]

md7 : active raid1 sdb7[1] sda7[0]
2096384 blocks [2/2] [UU]

md8 : active raid1 sdb8[1] sda8[0]
2096384 blocks [2/2] [UU]

md1 : active raid1 sdb10[1] sda10[0]
714752 blocks [2/2] [UU]

md11 : active raid1 sdb11[1] sda11[0]
2433728 blocks [2/2] [UU]

md2 : active raid1 sdb9[1] sda9[0]
2096384 blocks [2/2] [UU]

unused devices:

We see the MD device name: md and a device number, the state (‘active’), the RAID type (‘raid1′) and the actual devices the md device is build from (sdb1[1] sda1[0] for example). On the next line the number of blocks in the md device, the total number of disks and the number of active disks [2/2], and the status of the disks [UU]. “U” means up. A failed device (probably as a result of a failed disk, but it could be the result of the clearing of a partition) is visible as “_”.

Okay, this is all strongly linux related, where’s the Exadata stuff?

‘Cell’ is a name used for the storage server in the Exadata context. Both the computing nodes and the storage nodes have a directory ‘/opt/oracle.cellos’. This directory contains Oracle software which is embedded in Linux. The function of cellos is to maintain and check firmware versions and hardware configurations and hardware activation/deactivation.

Validation functions are done by ‘cellos’, or ‘scripts inside the /opt/oracle.cellos directory’ during startup phase of linux. One of the functions cellos can perform is flashing firmware. Strongly related to cellos are the linux kernel and the OFED stack.

The state of the cellos image is listed using the imageinfo utility (which exists both in /usr/local/bin and in /opt/oracle.cellos), which executes the /opt/oracle.cellos/patch/bin/ script:

On a database/computing node:

[root@xxxxdb01 bin]# imageinfo -all

Kernel version: 2.6.18- #1 SMP Tue Aug 31 22:41:13 EDT 2010 x86_64
Image version:
Image created: 2010-12-09 10:06:22 -0800
Image activated: 2011-01-28 14:19:59 -0800
Image image type: production
Image status: success
Internal version:
Image label: OSS_11.
Node type: COMPUTE
System partition on device: /dev/mapper/VGExaDb-LVDbSys1

On a cell/storage node:

[root@xxxxcel01 bin]# imageinfo -all

Kernel version: 2.6.18- #1 SMP Tue Aug 31 22:41:13 EDT 2010 x86_64
Cell version: OSS_11.
Cell rpm version: cell-

Active image version:
Active image created: 2010-12-09 09:37:46 -0800
Active image activated: 2011-01-28 14:33:44 -0800
Active image type: production
Active image status: success
Active internal version:
Active image label: OSS_11.
Active node type: STORAGE
Active system partition on device: /dev/md5
Active software partition on device: /dev/md7

In partition rollback: Impossible

Cell boot usb partition: /dev/sdac1
Cell boot usb version:

Inactive image version: undefined
Rollback to the inactive partitions: Impossible

We see imageinfo knows if it’s run on a computing or storage node (‘Active node type’).

Another executable in /opt/oracle.cellos is ‘CheckHWnFWProfile’. The name of this executabe is quite self-explanory: it checks if the configuration of the server it’s run on is valid, which means it is listed in one of the profiles (Exadata has different incarnations with different hardware, of which Oracle calls the hardware and firmware combination a ‘profile’).

If it’s run without any option, it should return [SUCCESS], which means the hardware and firmware are correct for that version of cellos:

[root@xxxxdb01 ~]# /opt/oracle.cellos/CheckHWnFWProfile
[SUCCESS] The hardware and firmware profile matches one of the supported profiles

If you want more information about the server, you can list the configuration with the ‘-d’ option:

[root@xxxxdb01 ~]# /opt/oracle.cellos/CheckHWnFWProfile -d
American Megatrends Inc.
Intel(R) Xeon(R) X5670 2.93GHz
step: 2
[ILOMVersion] r58740
PCI-E Slot 0
LSI MegaRAID SAS 9261-8i
PCI-E Slot 2
FM0 FM1 FM2 FM3 4
82599EB PCI-E Slot 1
82599EB PCI-E Slot 1
Firmware Revision : 3.0

If ‘CheckHWnFWProfile’ resulted in an error, you can list your actual configuration with ‘-d’, and use the ‘-s’ option to list what the profile or supported configuration is.

Tagged: database machine, exadata, imageinfo, infrastructure, oracle

Simple SQL with and without Inline Views 2

October 26, 2011 (Back to the Previous Post in the Series) In the previous article of this series we examined a couple of moderately simple data retrieval requirements, and used inline views and/or analytic functions to solve those data retrieval requirements.  I recently saw another opportunity to help a person on the ERP mailing list with [...]

OTN APAC Tour: Beijing, China (update)

Following on from my previous post, once the OTN APAC event was over I managed to see some of the sights of Beijing and the surrounding area, including the Forbidden City, Tiananmen Square, the Great Wall and Ming Tombs. I’ve uploaded some photos here, but it seems my photography skills are getting worse, not better. I’m ashamed to say these are the better photos. Let’s not discuss the ones I’ve not bothered to upload. :)

I have a few suggestions for any travellers to China:

  • Do not consider driving yourself. The traffic is crazy! I’ve never seen anything like it before. I experienced several “we’re all going to die” moments in every trip I took. Probably the worst was when two lorries moved together with our minibus between them. If our driver had not sped up in time, I’m quite convinced the ACE program would have lost several members. As you may have already read, some of the ACEs were in a crash, but came out of it unharmed. I was only able to wear a seat belt on one journey because all but one of the vehicles either had no belts, or belts but nowhere to plug them in. This takes a lot of getting used to.
  • If you have any sort of respiratory issues, it’s probably best to avoid Beijing completely. The pollution is very bad. It quickly makes you feel like you’ve smoked 40 cigarettes for breakfast. Even at the Great Wall (50+km from the city) it is still really bad.
  • Make sure you have printed copies of the Chinese names for any locations you want to visit. Having an English address is pretty much useless. Even with the Chinese addresses, it was often very difficult, especially if the print was small.
  • Don’t rely on any cloud services for your journey. You need local copies of everything and any apps you want to work with should have full offline functionality. Why? The Great Firewall of China is very effective at blocking a variety of services. Most Google services were trashed. Gmail was intermittently offline and I was never able to access attachments. Google Docs essentially doesn’t exist. Google Reader would work for about 5 minutes at a time, then fail bigtime. Most annoyingly, I was unable to use Google to search for anything. :( I was able to search with Bing, but I could only see results in Chinese. Even when I switched to English, I still got results in Chinese. Like I said, be very well prepared because searching for information is not easy. I’m sure there are ways to cope if you are clued up, but I was a little naive and went with very little preparation as far as logistics were concerned.
  • Take money with you. I was able to get cash advances from my credit cards, but the ATM failed to get cash from any of my bank accounts.

I’m now stationed at my friends house about 40 minutes from Auckland. There are cows in the garden, including 6 new calves which are pretty funny, and two pre-school children who are pretty funny too. Although I’m very much a towny at heart, I must admit the change in pace and air quality is very welcome. Last week I was talking about Oracle and this week I’ve chopping down some trees, cleared some scrub and pretended (very badly) to be a farmer. I’ve nearly mastered saying, “Get off my land!”, or as the Kiwi’s say it, “Gt ff my lnd aye!” I’m trying to introduce vowels to the natives, but with the exception of the word “aye” that is added to the end of every sentence, vowels seem to be lost forever. :)

In a few days I will transplant to Auckland and start the NZOUG conference.



Forms is not dead!

The recent interest in Oracle Forms is quite interesting. Besides ODTUG (Oracle Development Tools User Group); two other groups have recently conducted surveys to see how people are using forms one in Germany ( and one in the US ( Now, Oracle has release Oracle Forms 11g R2! First announced in Grant Ronald’s blog ( you may now find complete information and download the product from OTN’s Forms page ( I’ve just submitted a paper on the new version to ODTUG KScope 12 in San Antonio; I hope the paper is accepted!

While I work with many languages and tools; Forms is a very capable development tool and I’m looking forward to many more years of using it.

NoCOUG’s 100th Conference Nov 9th!

Download the 100th issue of the NoCOUG Journal at

Can you believe it? 100 conferences! The Northern California Oracle Users Group (NoCOUG) is celebrating its 25th anniversary in 2011 with its 100th conference. The upcoming conference at the Computer History Museum on November 9 is NoCOUG’s 100th quarterly conference and the upcoming issue of the NoCOUG Journal will be the 100th issue.
NoCOUG Conference 100 will feature some of the world’s top Oracle experts and speakers including Steven Feuerstein , Craig Shallahamer, Alex Gorbachev, and Kyle Hailey . It will be held at the Computer History Museum in Mountain View—a fitting location for such an occasion. The museum features marvelous computing artifacts such as a Hollerith Tabulating Machine and an actual operational Babbage Difference Engine—one of two that have been constructed in the past decade.
The 100th issue of the NoCOUG Journal will feature an interview with Michael Stonebraker—the high priest of relational databases, the research paper by Dr. Edgar Codd that started the relational revolution in 1970—A Shared Model of Data for Large Shared Data Banks, and the results of the Second International NoCOUG SQL Challenge.
Because of the significance of the occasion and the venue NoCOUG is expecting serveral hundred people that this conference will quickly reach maximum capacity, so RSVP now here:

RSVP by October 31 at to receive a free pass to the exhibition at the Computer History Museum.



What the heck is the SQL Execution ID – SQL_EXEC_ID?

Ok, I think it’s time to write another blog entry. I’ve been traveling and dealing with jetlag from 10-hour time difference, then traveling some more, spoken at conferences, drank beer, had fun, then traveled some more, trained customers, hacked some Exadatas and now I’m back home.

Anyway, do you know what is the SQL_EXEC_ID in V$SESSION and ASH views?

Oh yeah, it’s the “SQL Execution ID” just like the documentation says … all clear. Um … is it? I’d like to know more about it – what does it actually stand for?! Is it session level, instance level or a RAC-global counter? And why does it start from 16 million, not 1?


SQL> SELECT sql_exec_id FROM v$session WHERE sid = USERENV('SID');



This number 16777216 looks strangely familiar – indeed, it’s 2^24.

When I run the same query again (incrementing the SQL_EXEC_ID counter for the same SQL), I see the counter going up by 1:


SQL> SELECT sql_exec_id FROM v$session WHERE sid = USERENV('SID');


SQL> SELECT sql_exec_id FROM v$session WHERE sid = USERENV('SID');


SQL> SELECT sql_exec_id FROM v$session WHERE sid = USERENV('SID');


SQL> SELECT sql_exec_id FROM v$session WHERE sid = USERENV('SID');



Further executions of the same query keep incrementing this counter, one by one – even if I run this same SQL from another session in the same instance. So, this SQL_EXEC_ID is not a session-scope value for each SQL_ID, it’s at least instance-wide. It looks like the counting starts from 2^24 (the bit representing 2^24 is set) and ignoring that bit for now, the counting works normally, one by one, starting from zero.


Note that changing even a single character in the SQL text (see the extra space in the end before the semi-colon) causes the SQL_ID to change and a different SQL_EXEC_ID counter to be reported (which starts from “zero” again). A separate SQL_EXEC_ID counter is maintained in shared pool for each SQL_ID:

SQL> SELECT sql_exec_id FROM v$session WHERE sid = USERENV('SID') ;



So, obviously, when I have just restarted my instance and still see 16777216 as the starting SQL_EXEC_ID for any SQL I execute, it must mean that the full SQL_EXEC_ID value contains something else than just the execution number of this SQL_ID. Whenever I see such familiar values (like powers of 2), then I like to look into the values in hex format to see whether some higher order bits are used for some special purpose. Let’s run a new SQL statement:


SQL> SELECT sql_exec_id, TO_CHAR(sql_exec_id,'XXXXXXXX') hex FROM v$session WHERE sid = USERENV('SID') ;

----------- ---------
   16777216   1000000

SQL> SELECT sql_exec_id, TO_CHAR(sql_exec_id,'XXXXXXXX') hex FROM v$session WHERE sid = USERENV('SID') ;

----------- ---------
   16777217   1000001

SQL> SELECT sql_exec_id, TO_CHAR(sql_exec_id,'XXXXXXXX') hex FROM v$session WHERE sid = USERENV('SID') ;

----------- ---------
   16777218   1000002


Indeed, it looks like the 25th bit (2^24) is always pre-set to 1, while the least significant 24 bits represent how many times this SQL ID has been executed in an instance (I have tested this with a loop – the 24 least significant bits do get used fully for representing the SQL ID’s execution count in the instance and once it reaches 0xFFFFFF – or 0x1FFFFFF with that pre-set 25th bit, it wraps to 0×1000000 – the 25th bit still remaining set!). So the SQL_EXEC_ID can reliably only track 2^24 – 1 SQL executions in an instance and then the counter wraps to beginning. This is why you should include SQL_EXEC_START (date datatype with 1 sec precision) column in your performance monitoring queries as well, to distinguish between SQL executions with a colliding SQL_EXEC_ID. As long as you’re executing your SQL statement less than 16.7 million times per second per instance, this should be fine :-)


Anyway, so what’s the magic 25th bit then? Well, in RAC it would be very hard to somehow coordinate the incrementing of a single counter globally (that’s why you want to keep your sequences cached in RAC), I figure that there are different counters for the same SQL ID in different RAC instances. Let’s check – I will log in to another RAC node (node 2) and run this:


SQL> SELECT sql_exec_id, TO_CHAR(sql_exec_id,'XXXXXXXX') hex FROM v$session WHERE sid = USERENV('SID') ;

----------- ---------
   33554433   2000001

SQL> SELECT sql_exec_id, TO_CHAR(sql_exec_id,'XXXXXXXX') hex FROM v$session WHERE sid = USERENV('SID') ;

----------- ---------
   33554434   2000002

SQL> SELECT sql_exec_id, TO_CHAR(sql_exec_id,'XXXXXXXX') hex FROM v$session WHERE sid = USERENV('SID') ;

----------- ---------
   33554435   2000003


Whoa – the SQL Execution ID in the 2nd instance starts from 33 Million! And when you convert the value to hex, you’ll see that now the 26th bit is set – showing that this SQL was executed in instance #2!

So, it very much looks like that while the 24 least significant bits are used for the SQL execution ID counter, the more significant bits are used for showing which instance_id ran that SQL. Assuming that 32 bits are used for the whole SQL_EXEC_ID value, then up to 8 higher order bits could be used for storing the instance_id – supporting up to 256-node RAC clusters. This is very useful when analyzing past ASH data as you can aggregate data (count min/max exec ID difference to get the execution counts in a time range) either in each separate instance or globally – by stripping out the instance_id part from the value.

I haven’t tested the instance_id part with 256-node RAC clusters (as Santa Claus is cutting back due to poor economy), but at least on an 8-node full rack Exadata all 8 instance_ids were reported properly. Note that for serial queries, the SQL_EXEC_ID shows you the instance_id of the instance where the session is logged on to, but for inter-instance parallel query, you will see the instance_id of the query coordinator for all PX slaves, regardless of in which instances they run. Here’s a little script from a 8-node Exadata cluster to show it. I’ll leave it up to you to fully figure it out what, how and why it’s doing, but basically what it shows is that the SQL_EXEC_ID consists of the query coordinator’s instance_id value and the execution number for a SQL_ID in the instance where the query coordinator session was logged in:


SQL> SELECT qc_instance_id, MIN(TO_CHAR(sql_exec_id,'XXXXXXXX'))
  2  , MAX(TO_CHAR(sql_exec_id,'XXXXXXXX'))
  3* FROM gv$active_session_history GROUP BY qc_instance_id order by 1
SQL> /

-------------- --------- ---------
             1   1000000   100540F
             2   2000000   20009BF
             3   3000000   300541E
             4   4000000   40000DD
             5   5000000   50C5035
             6   6000000   600018C
             7   7000000   700023D
             8   8000000   8000755
                 1000000   803DF3B

9 rows selected.


That’s all for today – more cool stuff is coming, I promise :-)


And oh, next week I’ll start another run of my Advanced Oracle Troubleshooting seminar, so check it out! ;-)

Xplan utility

A utility to add parent ID and execution order information to plans reported by DBMS_XPLAN. XPlan includes DISPLAY, DISPLAY_CURSOR and DISPLAY_AWR functionality for use in exactly the same way as the DBMS_XPLAN equivalents. Supports versions from 10g onwards. ***Update*** Now available in two formats: 1) as a PL/SQL package and 2) as a collection of three free-standing SQL*Plus scripts (i.e. no installation/database objects needed). January 2009 (updated October 2011)

Simple SQL with and without Inline Views

October 21, 2011 (Forward to the Next Post in the Series) Sometimes it is interesting to take a step back from some of the more difficult to construct SQL statements, and just put together something simple.  The following request recently arrived through an ERP mailing list: “I want to find the last ship date for all the [...]

Friday Philosophy – Should I Be a Twit?

Something I have been pondering for a while now is should I join in with the “happening crowd” and sign up to Twitter? I know, I’m two or three years behind the times on this, but more and more people who I like have signed up – even Doug Burns now uses twitter and he used to be negative about it in the same way as I. I’ve asked a few of these friends what they think.

I’ve always resisted the whole micro-blogging world, probably due to the comments made by people about how much dross is tweeted and what a time sink it can be, something the people I asked all mentioned. And partly, being candid, because I know one of my faults is to shoot my mouth off before engaging my brain, especially if annoyed. If it takes 20 seconds to do a tweet, I shudder to think some of the things I might have put out there in the heat of the moment or when in the pub. Pub talk is fine, so long as it is kept in the pub. With a blog, it takes me so long to write them I tend to calm down before sending, if I am angry.

Also, tweets are not my style. I don’t know if you have noticed but I can be a little verbose {this means I spout several sentences where 5 words would do}. Can I be succinct enough to say anything anyone else would want to read?

That’s the writing of tweets, what about the following? I could just be a passive follower. But how many? This is part of the advice given to me by Neil Chandler and also Doug, to only follow a few people so that you do not just drown in an endless feed of stuff. I guess that anything good gets re-tweeted and so you see stuff not just by those you follow but the best bits of what they follow? Neil also suggested that part of what makes twitter work is joining in, don’t just be passive.

Something about Twitter that does bug me a lot, and this is just part of the whole texting/smartphone/constant communication thing of modern life, is when people you are spending social time with just sit staring at their bloody phone. I just find that really rude and I also think it’s a bit depressing when you see three or four people in the pub or restaurant, all staring at their smartphones and not communicating with each other. You might as well stay in bed. Alone. {Oh good grief what a terrible thought! Do couples who do social media now just sit in bed with their phones in front of them, ignoring their supposed love of their life?}

But of course there are advantages to Twitter. There is a lot of interesting stuff that goes on only in the twitter world and some of the tweets I have seen have been hilarious. It’s far more lightweight than blogging, something you can do in a quiet moment. Though Doug suggests this is why you get so much dross about travelling, people catch up on twitter when bored and also tweet then. Niall Litchfield made a very interesting point to me in that it allows you to keep up on a large range of topics and see new ideas and thoughts more. I like that. In fact, Niall sent me an excellent list of reasons for and against, which I’ll {almost} finish on.

From Niall:


Reasons for :

- I get vicarious updates from a large number of sources on a large number of subjects. (Oracle,SQL,Science,Politics)
- It can suit my short and acerbic style from time to time.
- More people are doing it
- It indirectly promotes me (albeit with a somewhat misleading image)
- I find stuff I wouldn’t otherwise have found
- I have engaged with product managers I probably wouldn’t have done

Reasons not:
- time wasting
- addictive
- it promotes me in a misleading way
- it can annoy others

I like it because I’m an information junkie and like to keep abreast of stuff in several areas. It can be a colossal waste of time and irritate immensely.


I think I might well give it a go and that leads to my last thought. If I am going to enter the world of twitter, I am going to have to get a new phone. My current one is so basic that it does little more than just make and receive calls. But a single charge lasts a week.

Tuning Blog Entries

Organizing  some of my past blog entries on Oracle database performance tuning and analysis. I’ll add more notes as well as a separate section on SQL tuning in particular.






VST – vistual sql tunning