Oracle And Open Source

Read an Excerpt

Chapter 1: Oracle Meets Open Source

• Introduction to Open Source
• Open Source Summary
• The Open Source Definition

The combination of Oracle Corporation and open source software may appear to be an unlikely pairing. What could Oracle, with its history of ruthless competition, intense marketing, and cutthroat corporate life, have to do with the collaborative, altruistic, and apparently anti-corporate world of open source?

The answer, surprisingly, is quite a lot. In recent years, the gospel of the open source movement has spread far and wide, reaching even the corporate corridors and product lines of organizations like Oracle Corporation. Consider the following recent developments:

• Oracle8i has been officially ported to the freely available Linux operating system.
• The open source Apache web server is now distributed as part of the Oracle Internet Application Server (iAS).
• The open source Perl, Tcl, and Python scripting languages all provide modules supporting connections to Oracle databases.
• The Oratcl application, an open source program built on the Tcl scripting language, is now distributed as part of the Oracle Enterprise Manager (OEM) product.
• Dozens of excellent applications written by open source developers--Orac, Oddis, Karma, Oracletool, OraSnap, Big Brother, jDBA, GNOME-db, and many more--give Oracle database administrators and developers new tools for managing their databases and building new applications. And if one of these tools doesn't do exactly what's needed in a specific environment, the source code can be modified without restriction.

In our opinion, this new synergy between the corporate world of Oracle and the freewheeling world of open source is a great thing--the blended products we're starting to see truly do represent the best of both worlds.

The purpose of this book is to share what we know about this blending of Oracle and open source. It's a new world, and one that hasn't been examined much to date. Although many books have been written about some of the base open source technologies we'll be exploring in this book (for example, Perl, Tcl, and Apache), there has been very little written about the way these technologies are used with Oracle databases, and even less about most of the Oracle open source applications available to DBAs and developers. Although we can't possibly describe every open source technology in depth within the confines of this single volume, we'll try to provide a foundation. Our overriding goal is to shed light on as many of the open source technologies and applications as possible and on how they communicate with Oracle databases. In trying to achieve this goal, we'll weave together three distinct threads:

• We'll explore the major open source technologies on the current computing landscape--Perl, Tcl, Python, Apache, GNOME, GTK+, and even some open source corners of Java--and explain how they are being used to build Oracle applications and connect to Oracle databases.
• We'll describe the best of the Oracle applications that are currently available for you to download, use, and, if you wish, modify. Some of these applications are tools for performing database administration tasks; others provide frameworks for application development of your own.
• We'll try to motivate you to consider writing your own Oracle open source applications in the future.

The world of open source software is a wide-ranging one. In this book, we'll focus on the open source technologies that are most often used by Oracle developers:

Scripting languages

We'll focus on Perl, Tcl, and Python, the most popular languages and the ones providing the most solid connections to Oracle databases. Chapter 2, Connecting to Oracle, and Chapter 3, Tcl, Perl, and Python, describe how to obtain and install the base languages, how to use their companion graphics toolkits, and how they communicate with Oracle. Chapter 4, Building Oracle Applications with Perl/Tk and Tcl/Tk, describes two excellent open source applications: Orac, built on Perl, and Oddis, built on Tcl. Both are good examples of how you can build Oracle-based graphical user interfaces (GUIs) with open source scripting languages and toolkits.

The Web

We'll next look at an alternative to GUI applications--Oracle applications that use the Web, rather than a GUI, as the user interface. We'll focus on the Apache web server, CGI programming with Oracle, and the use of such embedded scripting tools as PHP and EmbPerl. Chapter 5, Web Technologies, describes these open source technologies and how they connect to Oracle databases. Chapter 6, Building Web-Based Oracle Applications, describes an assortment of excellent web-based applications that can be used for Oracle database administration: Karma, Oracletool, OraSnap, DB_Browser, PhpMyAdmin/PhpOracleAdmin, WWWdb, and Big Brother.

Java

You may be surprised to see any coverage of Java in this book. Java is, after all, still a proprietary language of Sun Microsystems. Many open source developers are now building Oracle-based applications in Java, however, and Oracle is providing extensive support for Java connectivity. Chapter 7, Java, describes the basics of Java DataBase Connectivity (JDBC), which is used to connect programs to Oracle databases. It also describes the open source JDOM servlet API and some other Oracle-related aspects of Java, including the Apache JServ web server, now distributed with Oracle's Internet Application Server, and how you can use it to build Oracle Enterprise Java servlets. Chapter 8, Building Oracle Applications with Java, describes several excellent Oracle open source applications based on Java: jDBA, ViennaSQL, and DBInspector. It also explains how you can obtain and install DB Prism, a servlet-based tool for use with Oracle's PL/SQL and XML code facilities.

GNOME and GTK+

GNOME is an ambitious project aimed at creating a complete, open source desktop environment for Linux. Now that Oracle8i has been ported to Linux, GNOME and its excellent GTK+ graphical toolkit are available to Oracle developers. Chapter 9, GNOME and GTK+, describes the basics of GNOME and GTK+ and how you can use them to build graphical applications for Oracle. Chapter 10, Building Oracle Applications with GNOME and GTK+, describes the excellent Orasoft applications suite, a full-functioned database administration toolkit for Oracle DBAs, as well as several other GNOME/GTK+ applications that communicate with Oracle databases: GNOME-DB, gASQL, Gnome Transcript, and Gaby.

Although the sequence of chapters is roughly chronological (Perl and the other scripting languages were available before the coming of the World Wide Web, which preceded the development of Java and GNOME/GTK+), all four basic areas are still vibrant and fertile areas of open source activity, and all are focal points for Oracle application development. To a large extent, the choice of which open source technologies to use for developing an open source application is simply personal preference. Do you want to write in Perl, C, or Java? Do you want your user interface to be GUI or web-based? Fortunately, regardless of your choice, there is excellent Oracle connectivity available with all of the technologies we describe in this book. In subsequent chapters, we'll focus on this connectivity. As interesting and wide-ranging as these core technologies are, we'll try to limit our discussion to what you need to know in order to use them with Oracle, in an effort to keep this book a manageable size.

We've mentioned a lot of open source programs already, and we'll describe even more in this book. Table 1-2, included near the end of this chapter, provides a full list of the programs described, along with URLs for further exploration. (We'll also try to keep the URLs current at the O'Reilly web page for this book--see the preface for details.)

Introduction to Open Source

Before we get into the details of how you can use and build Oracle open source software, let's take some time exploring what open source is and why you might want to use it.

What Is Open Source?

The quick and dirty definition of "open source" is that it is software that's freely available: you can acquire it freely (it's usually downloaded from the Internet at no cost), and you can modify it freely (the source code is provided, not just the executable files). However, it's important that we refine this quick use of the word "free." There are important semantic differences between what the Free Software Foundation (FSF) defines as "free" and what the newer Open Source Initiative defines as "free." We'll explore both movements and their respective definitions later in this chapter. For starters, we'll just note the functional definition of "open source" from the Open Source Initiative web site, at http://www.opensource.org:

Open source promotes software reliability and quality by supporting independent peer review and rapid evolution of source code. To be OSI certified, the software must be distributed under a license that guarantees the right to read, redistribute, modify, and use the software freely.

The OSI web site continues with a description of why open source is of such high quality:

The basic idea behind open source is very simple. When programmers on the Internet can read, redistribute, and modify the source for a piece of software, it evolves. People improve it, people adapt it, people fix bugs. And this can happen at a speed that, if one is used to the slow pace of conventional software development, seems astonishing. We in the open-source community have learned that this rapid evolutionary process produces better software than the traditional closed model, in which only a very few programmers can see source and everybody else must blindly use an opaque block of bits.

A little later in this section, we'll discuss the open source license and the details of what it means to be OSI certified. For now, let's ask a few more questions.

Many people, accustomed to the traditional world of commercial software, are puzzled by the whole notion of open source. Why would anyone give away something of obvious value? Because they don't understand the motivation, they tend to distrust the product. They ask:

• How can open source software be any good if its author doesn't value it enough to charge for it?
• How can it be trusted to perform if it doesn't have the weight of a business behind it?
• How can it be responsive to users' needs if users don't have any contractual claim on it?

By even asking these questions, though, people are ignoring the clearest success stories of open source: Apache, the world's dominant web server; Perl, the world's dominant scripting language; and Linux, the world's most rapidly growing operating system kernel. All of these pieces of software have already been enthusiastically accepted by the corporate world as stable, successful, and, if not the best solutions in their fields, then at least major challengers. Many people seem to have a blind spot; they assume that because this software has, in fact, received commercial acceptance, it therefore can't be open source. But Apache, Perl, Linux, and the many other excellent pieces of software described in this book are indeed open source, through and through.

In the next few sections we'll look briefly at what open source is all about and why you, as an Oracle DBA or developer, might find it useful.^[1] Then we'll shift our focus to the main open source technologies available today, how they've been used to communicate with Oracle databases, and how you can use them to build your own Oracle-based applications.

Why Open Source?

Considering the enormous amount of software that's available to you from Oracle Corporation and various other software providers, why should you, as an Oracle DBA or developer, care about open source software?

Open source software is free in several senses of the word. It is typically free of cost, which means that you don't have to pay for it. It is also free of code restrictions, which means that the source code is provided, and you are free to modify it. Let's look at these two characteristics in turn and consider their relevance to your life with Oracle.

Oracle DBAs and developers typically work in corporate environments. The stereotypical user of open source software is the guy in the garage, with no salary, no budget, and no deep corporate pockets. The typical Oracle DBA isn't in that category; he often has a reasonably generous budget for software. What he might not have, however, is the flexibility to use that budget as he sees fit. If you need a piece of software right now, but you can't buy it without going through a lengthy approval process, then those corporate pockets won't seem so deep after all. You might as well be in the garage.

We're overstating the point, of course, but clearly the ability to simply download and try out many different possible tools before settling on one is enormously beneficial.

The second characteristic of open source, the ability to customize the source code to suit you, is often even more important. Most Oracle DBAs and developers are in a hurry. They are constantly fighting fires and trying their best to support the disparate needs of the users in their organizations.

The rapid advancement of technology, combined with the ever-increasing needs of an interconnected world, make the life of an Oracle DBA or developer more complex than ever. At one time, a DBA might have been able to do her job using a single, straightforward administration tool, combined with a small system of character-based entry forms. No more: the need for 24 × 7 availability, distributed processing, data warehousing, and ever faster response times has led to a bewildering array of products and responsibilities. Web connections, telecom billing, secure payments, and a host of other problems abound. It is the lucky Oracle DBA these days who is not weighed down with a GSM mobile, a pager, and a Palm Pilot filled with difficult assignments scheduled for particularly unsociable hours, or the lucky Oracle developer who has already mastered all the latest web development tools, Java servlets, and XML data parsers. Every site is different, and every user's problem is a new one. An enormous advantage of open source software is that if it doesn't do what you need, you can adapt it.

The Motivation Behind Open Source

Although "open source" is a relatively new term,^[2] its origins date back to the early days of computer software. And the more general concept of gaining value by giving value is one that predates computers themselves.

Most monetary economies in the corporate world are driven by some type of control mechanism or exchange mechanism (or, to be less technical, by the carrot and the stick). Let's suppose that you are a dedicated programmer working for largeCorporationWidgets.com, and your boss tells you to code up an Oracle Forms program or set up a database. You generally do as you're told. Why? Because you have previously agreed to be under the control of the company in return for various company benefits. The "stick" side of the equation is that if you fail to carry out an assigned task, you won't be paid and you may lose your job. On the "carrot" side, you will receive a salary in payment for doing your job, and eventually you may earn a promotion.

Until recently, these complementary rationales were the only generally accepted methods for driving innovative software creation (and for motivating dedicated programmers, most of whom have the natural herding instincts of paranoid tigers). However, hiding behind this control and exchange mechanism are two implicit assumptions:

1. largeCorporationWidgets.com controls all access to computing power and the network. If they fire you, you will no longer be able to write programs or communicate with any of your programming friends around the world. If you're a dedicated hacker,^[3] this may be important to you.
2. You need the money.

In contrast, let's look at an alternative, rather plausible universe in which these two assumptions don't necessarily hold true. Suppose you're a gifted hacker at MIT who is happy living upon a secure research grant; or you're a well-paid professor of computer science at Stanford who wants to attract the brightest undergraduates; or you're a dedicated nighttime hacker, who makes enough during the day (possibly supporting legacy systems for largeCorporationWidgets.com) to pay for the latest workstation and a 24-hour, high-speed Internet connection. In all of these cases, you find yourself with an abundance of computer-related material goods, networks, and processing power, plus enough money to eat and pay the rent. When it comes right down to it, you simply don't require any more in the way of material possessions (computer or otherwise). In none of these cases is there any outside force that can persuade you to work any harder.

Given that money isn't a sufficient motive in these cases, what then could motivate you to write those programs? There are a number of possible motives:

• The need to achieve the highest possible status among your peers
• Pure scientific curiosity and the need to write programs that are interesting and fun
• The need to solve problems that are hampering your ability to do your job (in the open source world, this is known as "scratching a developer's itch")

Let's look at these three motives in turn.

The gift culture

The gift culture is an anthropological term that Eric Raymond, a major force in the open source community (we'll discuss his role a bit later in this chapter), first used to describe the culture of open source. In the world of software, the idea of a gift culture is, basically, that if you give away cool programs that do useful things for other people, then your stock will rise among your colleagues. The degree of regard in which others hold you will depend exponentially upon the quality of your gifts. This last point is the prime engine that drives the open source movement. Raymond writes that, in a gift culture:

Participants compete for prestige by giving time, energy, and creativity away . . . In gift cultures, social status is determined not by what you control but by what you give away.^[4]

Thus, when Larry Wall gives Perl (essentially, the Mount Olympus of open source software) away to the hacker community, he rises to a transcendental plane occupied by only a few other deities.^[5] This pantheon is an extremely exclusive club, and no amount of money will buy you a membership card. You have to earn membership another way--by giving away the virtual crown jewels of your programming creativity.

The whole notion of the gift culture may seem strange to you, but note that the concept is not specific to the world of computing. Raymond writes:

Gift cultures are adaptations not to scarcity but to abundance. They arise in populations that do not have significant material-scarcity problems with survival goods. We can observe gift cultures in action among aboriginal cultures living in ecozones with mild climates and abundant food. We can also observe them in certain strata of our own society, especially in show business and among the very wealthy.^[6]

The need for innovation

There is a human need for progress, innovation, and building things of value that is by no means unique to the world of computing. Most scientists tend to gravitate to areas that intrigue them, for whatever reason, and a surprising number choose to focus on these areas even if there is little chance of immediate financial reward.

Consider Albert Einstein. Einstein spent many years studying on his own a topic of great interest to him: a comparison of James Clerk Maxwell's idea (circa 1865) that light traveled at a single relative fixed speed to Max Planck's alternative ideas (circa 1900) on constant light energy quanta. Einstein had ample opportunity to perform lucrative pre-World War I physics research for any one of the warlike governments of Europe, but he chose to stay ensconced within the quiet of his Swiss Patent Office in Berne. With no particular promise of financial reward, he was determined to resolve the problem of what would happen if you could travel upon Maxwell's constant-speed lightbeam in Planck's quantized universe.^[7]

In the world of computer software, we see a similar process. Although much innovation is directed by the promise of financial reward, a good deal of energy goes into developing software that is simply interesting to an individual developer. Some of that development leads to blind alleys, of course, but in the best cases, developers create wonderfully innovative programs that ultimately become virtual standards.

The developer's itch

There are a great many interesting technical problems to solve and a great deal of challenging software to be written. How do open source developers decide how to spend their time? Most often, they start out trying to solve a particular problem or speed up a process that is getting in the way of doing their work in the most efficient way.

As Eric Raymond puts it, "Every good work of software starts by scratching a developer's personal itch." An individual developer is unsatisfied with the capabilities of a particular piece of software, so he decides to extend the program to suit his own needs. Maybe it turns out that other developers can benefit from his work. Ultimately, that extension, or modification, gets added to the general community of programs, just as in traditional science, one scientist's new theory gets added to the canon of established science. The most functional and usable programs become the foundation stones for the next generation of software. Since so much open source software is built on what has come before, no self-respecting open source developer would even contemplate keeping his innovations secret or making them too expensive for his peers to acquire. Most such developers see themselves as being only the temporary torch bearers in an unending chain of runners. To fail to pass on the baton would be to forfeit all respect.

In a similar way, all scientists build upon the great ideas of the past. However creative and revolutionary Einstein's work was, it relied in part on work that had gone before. Without Maxwell's mathematical equations, published in the public domain for all to see, Einstein would not have been able to build upon them, and physics could have languished for a much longer time under the older Newtonian "optics" model. And, continuing in the scientific tradition, Einstein's work built a foundation for others to work on and modify in the future.

Sir Isaac Newton put it well. He spurned accolades for his many achievements and credited those scientists who had gone before him, saying: "I have stood upon the shoulders of giants."

A good example of this progression in the computing world is the Linux operating system, which we'll look at in somewhat more detail later in this chapter. Linus Torvalds took the ideas contained within Andrew Tanenbaum's Minix operating system and used them to create his own kernel. Torvalds' original motivation was an "itch," the desire to read Usenet news groups without becoming reliant upon the binary-only operating systems that at that time were the only systems available on PCs. Torvalds published his early efforts in 1991, and the Linux project rapidly grew to become the poster child for open source. However, Linux success would never have evolved as it did without the availability of the source code for Tanenbaum's Minix. And the many creative developments built on Linux--for example, the GNOME/GTK+ technologies we'll look at in Chapters and --would probably never have come into existence.

A Brief History of Open Source

It may be useful to take a look back in time to see how we got where we are today. The concept of open source didn't just arrive, without precedent, on the computing scene. In some ways, as we've seen with Einstein, the basic motivations are as old as science itself. However, you'll be relieved to know that we'll confine our journey to the world of computing. Obviously, in this short section we can only touch on many events that are more complex and interesting than we can possibly express here. We apologize for the necessary brevity and oversimplification.

The early days of artificial intelligence

To start our journey, let's go back all the way to Alan Turing in 1945. After helping break the German military codes (including the infamous Enigma) in World War II, Turing began working for the National Physical Laboratory (NPL) in Britain on a project known as the Automatic Computing Engine (ACE). Although ACE wasn't the success Turing had hoped for, his ideas on artificial intelligence (such as the famous Turing test) permeated the NPL. Turing had a strong influence on Donald Davies, whose 1966 "Proposal for a Digital Communication Network" would later play a crucial role in the development of the ARPAnet in the United States.

With some cross-fertilization between NPL and the Massachusetts Institute of Technology, Turing's ideas on AI finally took root in 1956 under the leadership of John McCarthy (the inventor of Lisp) and Marvin Minsky, and culminated in the founding of the MIT Artificial Intelligence Project in 1960. A decade later, in 1971, the AI Lab at MIT took on perhaps its most famous student to date, Richard M. Stallman, thereby setting in place a chain reaction of crucial events that culminated in the GNU/Linux operating system. Before we get there, however, we need to look back at a second historical thread.

The rise of the Internet

In 1958, the United States government created the Advanced Research Projects Agency (ARPA). ARPA gradually began to take on much of the "blue-skies" computer research for the Pentagon and the rest of the U.S. defense establishment. In 1962, J.C.R. Licklider arrived at ARPA from MIT and put in place the research processes for networking and time-sharing operating systems. By 1969, these systems had come together into the ARPAnet project, which was the direct lineal ancestor of today's Internet.

The key to the ARPAnet's revolutionary communications system was a distributed, fault-resilient network with multiply redundant nodes. The idea was that, even if many of these nodes were damaged, messages would still be able to get through via whatever path remained. This model, originally developed by Paul Baran, was initially rejected by AT&T (which at the time supplied many of the American government's communication requirements), because it did not match their more centralized analog systems. However, the NPL scientist Roger Scantlebury later rediscovered Baran's work and combined it with Donald Davies' research. The result was the basic notion of packet switching, a concept on which the entire Internet rests. The ARPAnet subsequently went from strength to strength, eventually evolving into today's Internet. Before we jump too far ahead, though, let's look at the third thread in the interwoven origins of open source.

The birth of Unix

AT&T originally resisted Baran's multiply redundant and distributed network. However, in 1969, at the Bell Labs province of the AT&T empire, Ken Thompson invented a new operating system called Unix (the name was allegedly a pun on the earlier MULTICS system). Together with Dennis Ritchie's new C language, Thompson went on to make the Unix operating system truly portable by adapting it for a wide range of machines and creating portable Unix-based programs via the C programming language. The fact that the source code for Unix was also openly distributed to a large number of universities and institutions (the University of California at Berkeley played a particularly important role) helped with the operating system's portability. This was especially significant at the time because AT&T had been blocked from entering the computer business by a U.S. government antitrust measure in 1956.

From 1974 on, staff at Berkeley and Bell Labs worked in close cooperation to improve the original AT&T System V Unix (mainly on the Digital Equipment Corporation PDP-11). The operating system quickly became a favorite of hackers because of its special approach: it started with simple robust tools, which fed into more complex ones via the Unix pipe concept. In 1977, Bill Joy (later of Sun Microsystems) put together the first so-called Berkeley Software Distribution of Unix (BSD), which he worked on until 1982, when he joined the fledgling Sun Microsystems....

Read More