Search

Top 60 Oracle Blogs

Recent comments

Oracle

Sneak Preview of pgio (The SLOB Method for PostgreSQL) Part IV: How To Reduce The Amount of Memory In The Linux Page Cache For Testing Purposes.

I hope these sneak peeks are of interest…

PostgreSQL and Buffered I/O

PostgreSQL uses buffered I/O. If you want to test your storage subsystem capabilities with database physical I/O you have to get the OS page cache “out of the way”–unless you want to load really large test data sets.

Although pgio (the SLOB Method for PostgreSQL) is still in Beta, I’d like to show this example of the tool I provide for users to make a really large RAM system into an effectively smaller RAM system.

Linux Huge Pages

Memory allocated to huge pages is completely cordoned off unless a process allocates some IPC shared memory (shmget(1)).  The pgio kit comes with a simple tools called pgio_reduce_free_memory.sh which leverages this quality of huge pages in order to draw down available memory so that one can test physical I/O with a database size that is quite smaller than the amount of physical memory in the database host.

Sneak Preview of pgio (The SLOB Method for PostgreSQL) Part III: Link To The Full README file for Beta pgio v0.9.

If you are interested in a head start on pgio, the following is a link to the full README file which has some loading and testing how-to:

The pgio text README file version 0.9 Beta

Sneak Preview of pgio (The SLOB Method for PostgreSQL) Part II: Bulk Data Loading.

Bulk Data Loading With pgio Version 0.9 Beta

Now that pgio (the SLOB Method for PostgreSQL) is in Beta users’ hands I’m going to make a few quick blog entries with examples of pgio usage. The following are screen grabs taken while loading 1 terabyte into the pgio schemas. As the example shows, pgio (on a system with ample storage performance) can ready a 1 terabyte data set for testing in only 1014 seconds (roughly 3.5TB/h loading rate).

$ cat pgio.conf
UPDATE_PCT=0
RUN_TIME=120
NUM_SCHEMAS=4
NUM_THREADS=2
WORK_UNIT=255
UPDATE_WORK_UNIT=8
SCALE=1G

DBNAME=pg10
CONNECT_STRING=”pg10″

CREATE_BASE_TABLE=TRUE

Sneak Preview of pgio (The SLOB Method for PostgreSQL) Part I: The Beta pgio README File.

The pgio kit is the only authorized port of the SLOB Method for PostgreSQL. I’ve been handing out Beta kits to some folks already but I thought I’d get some blog posts underway in anticipation of users’ interest.

The following is part of the README.txt for pgio v0.9 (Beta). SLOB users will find it all easy to understand. This is the section of the README that discusses pgio.conf parameters:

UPDATE_PCT

The percentage of SQL that will be UPDATE DML

RUN_TIME

runit.sh run duration in seconds

NUM_SCHEMAS

pgio data is loaded into either a big single schema or multiple. NUM_SCHEMAS directs setup.sh to create and load NUM_SCHEMAS schemas.

NUM_THREADS

For setup.sh:  This parameter controls the number of concurrent
data loading streams.

For runit.sh:  This parameter controls how many sessions will attach to
each NUM_SCHEMAS schema.

My Performance & Troubleshooting scripts (TPT) for Oracle are now in GitHub and open sourced

I have uploaded my TPT-oracle scripts to GitHub and have formally open sourced them under Apache 2.0 license as well. This allows companies to embed this software in their toolsets and processes & distribute them without a worry from legal departments.

The repository is here:

Now you can “git clone” this repository once and just “git pull” every now and then to see what updates & fixes I have made.

Also if you like my scripts, make sure you “Star” this repository in Github too – the more stars it gets, the more updates I will commit! ;-)

Bitmap Join Indexes

I’ve been prompted by a recent question on the ODC database forum to revisit a note I wrote nearly five years ago about bitmap join indexes and their failure to help with join cardinalities. At the time I made a couple of unsupported claims and suggestions without supplying any justification or proof. Today’s article finally fills that gap.

VMWare Experts Program, Sydney

A few weeks back, I received an invitation from Don Sullivan to attend the Sydney version of the VMWare Experts Program. I worked with Don during our time at Oracle, and caught up with him again a couple of years ago at one of the Collaborate conferences. He had moved on to VMWare, and is still working for them today.

VMWare Experts Program, Sydney

A few weeks back, I received an invitation from Don Sullivan to attend the Sydney version of the VMWare Experts Program. I worked with Don during our time at Oracle, and caught up with him again a couple of years ago at one of the Collaborate conferences. He had moved on to VMWare, and is still working for them today.

12c upuserxt.lst, upobjxt.lst & Oracle Maintained objects/users

Mike Dietrich has blogged recently about upuserxt.lst and upobjxt.lst and how to query them with external table. The first time I’ve seen those ‘.lst’ files, the default extension for sqlplus spool files, I wondered whether they were provided in ?/rdbms/admin on purpose, or if they were just some leftovers from some tests Oracle did before packaging the Oracle Home. Finally, I realized that they were there on purpose and that those ‘.lst’ are important files when upgrading to 12c.

Time for #GLOC, #SQLSatDallas, #DataSummit18

The next nine days, I’m traveling to three cities for four events. We’ll just call this the 9-3-4 gauntlet of speaker life. I booked this travel as four, one-way flights to get the itinerary
I needed to make the most of my schedule and will have breaks between each event to make sure I don’t kill myself my last two weeks at Delphix.

GLOC