Inside the Machine

Inside the Machine: A Journey into the World of High-Frequency Trading

10 Jun 2010

Michael Peltz

An editor's journey into the world of high-speed trading and proprietary algorithms that make or break markets.

At 2:45p.m. on Thursday, May 6, George (Gus) Sauter received a frantic call from one of his traders to get in front of a Bloomberg terminal. The Dow Jones industrial average, already down 3.9 percent that day on fears about Greece, was in free fall. In just five minutes the index plunged 573 points. Less than two minutes later, the Dow had rocketed back up 543 points, going on to finish the day down 3.2 percent.


“It was just crazy,” Sauter, chief investment officer of mutual fund giant Vanguard Group, told me a few days later. “I had to go to our fixed-income building, about a five-minute walk from my office. By the time I got there, the market had rallied.”

Crazy, indeed. The aptly named “flash crash” temporarily wiped out more than a half trillion dollars in equity value, shaking what little faith nervous investors had in U.S. markets. Shares of Dow component Procter & Gamble Co., the ultimate defensive blue-chip stock, dropped more than one third in a matter of minutes before recovering almost as quickly, all for no apparent reason. A few other large U.S. companies, including accounting firm Accenture, saw their stocks trade as low as a penny a share, only to close not far from where they had begun the day (nearly $42 a share in the case of Accenture) — again, on no news. By the time the dust settled, a whopping 19.3 billion shares had changed hands, more than twice the average daily U.S. equity market volume this year and the second-biggest trading day ever.

But for me, the single most amazing fact about the flash crash was that no one had a clue as to what had triggered it. Not that I should have been surprised, based on the conversation I’d had two days earlier with Mary Schapiro, chairman of the Securities and Exchange Commission. Schapiro, whose organization is charged with maintaining “fair and orderly” markets, explained to me how the SEC did a detailed study after the October 1987 crash to reconstruct what had happened. “We’ve lost some of the capacity to do that given the dramatic volumes of trading that exist today,” Schapiro said. “But we need to be able to do that to understand where are the vulnerabilities in our marketplace and what are the practices that have the potential to hurt investors and the marketplace in the long run.”

In 1987 the SEC had a much easier task because the vast majority of listed U.S. equities were traded in one place — on the floor of the New York Stock Exchange, where specialists employed by the Big Board’s member firms made a market based on an open-outcry auction system. Today, as a result of a series of regulatory changes designed to increase competition and make the market fairer for mom-and-pop investors, only about one quarter of all U.S. equity trading occurs through the now publicly held NYSE Euronext. And the majority of that trading is done electronically, either by the new NYSE floor specialists, called designated market makers, or on the fully automated NYSE Arca platform. The rest of the trading in U.S. equities is spread across a wide range of venues, including the three other major exchanges (Nasdaq Stock Market, BATS Exchange and Direct Edge) and dozens of broker-dealer-operated trading systems, electronic communications networks (ECNs) and dark pools, where buyers and sellers are matched up anonymously.

The past decade of fragmentation and automation has given rise to a whole new type of professional trading firm: one that uses sophisticated computer algorithms, often running on servers housed right next to exchanges’ own machines, and high-speed market data feeds to buy and sell securities in rapid-fire fashion. Some of these high frequency traders place hundreds of millions, even billions, of buy and sell orders a day, continually canceling and replacing them, and are likely to be on the other side of your trade. Not that you’d know who they are — proprietary trading firms are not required to disclose their identity — or recognize their names. The bulge bracket of high frequency trading includes firms like Allston Trading, DRW Holdings, Global Electronic Trading Co. (Getco), Hudson River Trading, Quantlab Financial, RGM Advisors, Sun Trading, Tower Research Capital and Tradebot Systems.

High frequency trading has become a multibillion-dollar business, accounting for an estimated 50 to 70 percent of the total U.S. equity market volume on any given day. Since last summer it has also been a lightning rod for the populist anger directed at Wall Street, despite the fact that most of the largest high frequency firms operate far from the canyons of lower Manhattan, in places like Chicago, Kansas City and Austin, Texas. Critics accuse high frequency traders of being fair-weather market makers who, unlike the former NYSE specialists they’ve largely replaced, don’t have a legal obligation to trade during periods of stress. They also say that the growth in high frequency trading has created a two-tiered market of technology haves and have-nots that is unfair to long-term investors and poses potential systemic risks.

I grew interested in high frequency trading last year when I was writing a feature on hedge fund firm Citadel Investment Group (more on that later). As an editor, however, it wasn’t until January that I was able to dig into what I soon learned is an incredibly arcane world. My first stop was a company called Pragma Securities, an agency-only brokerage firm that aggregates more than 40 different dark pools, electronic trading venues and open market destinations into a single liquidity source for clients. Douglas Rivelli and David Mechner, Pragma’s co-CEOs, spent two hours at the firm’s spacious New York offices taking me through that world.

High frequency traders, Rivelli and Mechner explained, generally fall into one of two camps: proprietary trading shops that act as electronic market makers, using computers to generate and adjust buy and sell orders automatically throughout the day, and hedge funds that specialize in statistical arbitrage, seeking to exploit pricing inefficiencies among different securities and asset classes. The distinctions between the two sometimes blur, however, as proprietary trading firms often try to capitalize on some of the same buy and sell signals that statistical arbitrageurs use and hedge funds trade on ever-shorter time horizons. High frequency firms are best known for trading equities, but they also trade futures, options and foreign exchange — basically, anything that can be traded electronically. High frequency trading is also an increasingly global phenomenon, gaining ground in both Europe and Asia.

One thing is clear: Hedge funds don’t like to be called high frequency traders, as I quickly discovered after visiting with some of the biggest quantitative managers, including AQR Capital Management in Greenwich, Connecticut, and D.E. Shaw & Co. and Renaissance Technologies Corp. in New York.

In the wake of the flash crash, as people scrambled to determine what had triggered the market plunge, it didn’t take much longer than the 400 to 600 microseconds (millionths of a second) that high frequency traders typically need to identify and place a trade for fingers to start pointing at them. “The potential for giant high-speed computers to generate false trades and create market chaos reared its head again today,” Delaware Senator Ted Kaufman said in a statement released that same afternoon. When I caught up with the Democratic lawmaker a week later, he was even more incensed, pointing out that regulators still didn’t know what had caused the flash crash.

“We have a 300-pound gorilla in the room, and we’re saying that we’re going to keep it in a cage somewhere,” he told me. “This thing will be 600 pounds.”

“But isn’t part of the problem that there are 300 gorillas?” I asked, referring to the fact that an estimated 200 to 400 firms do high frequency trading.

“Good point,” he replied. “We have all these gorillas, and guess what? We put them in zoos where the people running the zoos don’t have enough information and authority to take care of them.”

Kaufman’s interest in high frequency trading predates mine. When he was sworn into office in January 2009 to fill the Senate seat of his former boss Joe Biden, Kaufman was hell-bent on making sure that everybody responsible for the 2008 market meltdown paid for their actions. He soon focused on short-selling, urging the SEC to reinstate the uptick rule requiring short sales to be filled at a higher price; the rule had been eliminated in 2007. He told me that when he was in business school in the 1960s, it was “an article of faith” that the uptick rule was “one of the two or three things that helped deal with predatory bear raids.” As a result of his interest in short-selling, Kaufman said, his office started getting calls from some fairly sophisticated people, including former Wall Streeters, telling him that if he thought that practice was bad, he should look at high frequency trading.

Kaufman likes to draw an analogy between high frequency trading and the swaps market. “With synthetic derivatives, you had a lot of money at stake, no transparency and then a major meltdown,” he explained to me. “If you look at high frequency trading, I think the same Kaufman formula works.”

A graduate of the Wharton School of the University of Pennsylvania, the 71-year-old Kaufman is a quick study and understands markets. If I were a high frequency trader, I’d take him seriously.

Kaufman has been high frequency trading’s loudest critic. But he’s far from alone. Seth Merrin, founder and CEO of Liquidnet Holdings, which operates an electronic marketplace that provides block trading for institutional investors, likes to compare high frequency traders to the American army during the Revolutionary War. “The institutions are the equivalent of the British army, walking down the battlefield wearing bright red,” he told me back in March in his glass-enclosed office at Liquidnet’s sleek midtown Manhattan headquarters. “The high frequency traders are the Americans hiding in the woods in camouflage, picking them off. If the British army hadn’t changed its tactics, they would have lost every subsequent war.”

Even Duncan Niederauer, who as the pragmatic CEO of NYSE Euronext has been retooling his exchange to attract more business from high frequency traders, took a swipe at them. “We as an industry have to say how much is too much of this technology,” he said during an interview on CNBC after the flash crash, undoubtedly causing some consternation among the folks at NYSE Euronext who are selling space in the company’s new, 400,000-square-foot data center and co-location facility in Mahwah, New Jersey.

High frequency traders say that any efforts to rein in technology would be misplaced. Although speed is important to what they do, the quality of a firm’s computer models for analyzing markets and identifying where and at what price to buy and sell securities is what really determines success or failure, they argue. In their defense, high frequency traders say that they increase liquidity, lower trading costs, improve price discovery and reduce risk by dampening short-term volatility.

“High frequency trading is the liquidity backbone of the equity markets,” Manoj Narang, the founder, CEO and chief investment strategist of Tradeworx, told me when I first met him, in early March. “Long-term investors are the ones who cause bubbles, as well as liquidity crises when these bubbles burst.”

Narang, 40, is one of only a handful of proprietary traders I found willing to talk openly with a journalist about what they do. Most prefer to operate in the shadows, both to protect their valuable algorithms and to avoid regulatory scrutiny. But Narang, who left Wall Street in 1999 to start Tradeworx, sought me out when he heard through a public relations contact this winter that I was working on a story on high frequency trading. His 25-person firm, which operates out of an office above a Restoration Hardware store in Red Bank, New Jersey, trades about 40 million shares a day on about $6 million in proprietary capital. Tradeworx also runs a $500 million statistical arbitrage hedge fund (which trades another 40 million shares a day) and owns a subsidiary, Thesys Technologies, which licenses its high-performance trading platform to other investors.

Narang lifted his profile on May 6 when he revealed to the Wall Street Journal that his firm turned off its high frequency trading computers during the flash crash. Tradeworx wasn’t the only one to do so. Kansas City–based Tradebot, started by BATS founder David Cummings, also stopped trading. Tradebot is one of the world’s two largest high frequency firms, reportedly trading as many as 1 billion shares a day in U.S. equities. Only Chicago-based Getco is thought to be bigger. Although Getco won’t comment on its daily trading volume, a spokeswoman for the firm did tell me that it continued to provide a two-sided market on all the electronic exchanges during the flash crash.

Most high frequency traders, in fact, kept their computers running, according to Jeffrey Wecker, president and CEO of Lime Brokerage. Wecker should know. His firm, which accounts for as much as 5 percent of the daily equity trading volume in the U.S., is the oldest and largest provider of high-speed trading solutions and access to all major U.S. exchanges for high frequency traders.

The high frequency firms that did stop trading on May 6 have been criticized for contributing to the decline by pulling liquidity from the market when it was needed most. But Narang told me that his firm had no choice because the exchanges were likely to cancel, or break, trades that were clearly erroneous (like selling Accenture at a penny a share). “If the exchanges broke all our buys and not our sells, we could have exceeded our capital requirements,” he explained. “We didn’t want to take the risk. The high frequency traders who continued to trade that afternoon made a fortune.”

IN JANUARY THE SEC PUBLISHED A CONCEPT RELEASE on equity market structure, seeking public comment on everything from high frequency trading strategies and systemic risks to co-location and dark pools. At 74 pages, the report might seem like a real snoozer, but it’s actually a great primer on how the U.S. equity markets have responded to regulatory changes, starting in 1996 with the adoption of “order-handling” rules. These new rules, which were designed to make the markets fairer following the Nasdaq price-fixing scandal in the mid-’90s, created ECNs and gave them the power to publish their stock quotes publicly alongside those of the listed markets. In 1999, Regulation Alternative Trading System (ATS) went into effect, enabling ECNs to operate as market centers without having to register as exchanges. By the following year ECNs like Island and Archipelago had taken about one third of market makers’ volume in Nasdaq-listed stocks. But it wasn’t until after April 2001 — the deadline the SEC mandated for all U.S. exchanges to switch from fractions to decimals — that electronic trading really started to take off.

As bid-offer spreads shrunk and competition increased, ECNs and exchanges adopted a “maker-taker” pricing scheme to attract liquidity. Under the maker-taker model, market participants that offer to provide, or make, liquidity by posting an order to buy or sell a certain number of shares at a particular price receive a rebate. Those that execute against that order — that is, take the liquidity — have to pay a fee. Exchanges earn the difference between the rebate they pay and the fee they charge. The SEC limits taker fees to 0.30 cents a share; rebates tend to be lower for economic reasons, but for high frequency firms trading millions of shares a day, they can make for a pretty good living.

“The maker-taker model created an arbitrage that provided incentive for those firms that could properly blend together knowledge of trading, financial economics and computers all into a single, scalable system that could handle high volumes of transactions,” Lime Brokerage CEO Wecker told me back in April when we met at his firm’s Greenwich Village offices.

The final major regulatory change was Regulation NMS (for “National Market System”), which was passed by the SEC in 2005 and went into effect in 2007. One of the key pieces of Reg NMS is the “trade-through” rule prohibiting any exchange from executing a trade at an inferior price to one quoted at another trading venue. Trade-through protection is critical for high frequency traders; it ensures that if they are the first to post the best price for a stock, they’ll get the trade.
“High frequency trading is a product of Reg NMS, decimalization and technology improvements,” says John Knuff, general manager of global financial markets for Equinix, a Foster City, California–based company that runs 87 data centers in 35 key metropolitan areas around the world. “High frequency traders are the democratic enforcers of Reg NMS’s trade-through rules.”

Under Reg NMS, exchanges are required to handle electronic orders immediately or risk having them redirected to other venues. Once the rule was adopted, the NYSE — which even after decimalization had been clinging desperately to its manual specialist system — had no choice but to embrace automation. In 2007 the NYSE switched to a system it called the Hybrid Market, expanding its automatic execution facility, Direct+, and giving specialists the power to create their own algorithms to quote and trade electronically. The hybrid system included circuit breakers, called liquidity replenishment points, that would be triggered if a stock experienced a large price swing, at which time automated trading would stop and human specialists would step in.

That’s exactly what happened on the afternoon of May 6, when the NYSE imposed a trading slowdown in Big Board stocks like P&G and Accenture. Investors who wanted to sell or buy were forced to go to other electronic exchanges or ECNs. Although Nasdaq OMX Group CEO Robert Greifeld and other competitors criticized the NYSE on May 6 for making the meltdown worse, NYSE Euronext CEO Niederauer staunchly defended its actions, pointing out that the exchange did exactly what it should have under Reg NMS. He was vindicated two weeks later when the SEC proposed instituting similar circuit breakers for all exchanges that would pause trading in any stock in the Standard & Poor’s 500 index if its price moved 10 percent or more in a five-minute period. The new circuit breaker rule, which is likely to be approved by the SEC this month, would go into effect on a pilot basis through December 10, at which point the regulator could expand it to other stocks and exchange-traded funds.

“When a stock gets overheated, some form of stock-specific circuit breaker is a very effective means for letting information repopulate in the marketplace,” Lime’s Wecker told me a few days after the flash crash. He advocates an initial short-term cooling-off period measured in seconds or minutes; if a stock suffers a subsequent steep decline in price, the next halt would be longer.

Like many of the people I have interviewed about high frequency trading over the past five months, Wecker is concerned that regulators could try to rein in the practice by putting limits on technology. After all, his business is built on speed. “The challenge with speed bumps is that you are slowing down innovation to accommodate the players who have no interest in investing in innovation,” says Wecker, 47, who spent 11 years at Goldman, Sachs & Co., including six in its famed quantitative trading group, before eventually moving to Lehman Brothers to build its electronic trading group. He left Lehman in April 2008, five months before the investment bank filed for bankruptcy, and was hired by Lime founder Mark Gorton that November.

Lime handles hundreds of millions of trade orders a day. This spring it introduced a product called LimeInside that enables customers to place an order with Nasdaq, NYSE Arca and BATS in less than ten microseconds, on average — including real-time pretrade risk checks. That’s blazingly fast, confirms Tradeworx CEO Narang. It takes his group about 20 microseconds to do a trade from the moment a stock quote enters its system, triggers a signal, determines an order and passes through risk controls. Besides Lime, the only firms that are faster than Tradeworx, Narang says, are Tradebot and Getco.

Trading in the single-digit microseconds would be impossible for firms like Lime, Tradebot and Getco if they didn’t house their algorithms near the computer matching engines that power exchanges, ECNs and other electronic marketplaces. Brokerages, proprietary trading firms, hedge funds and other asset managers can lease co-location space in exchange-owned facilities (such as the NYSE’s Mahwah data center) or those owned and operated by third-party providers like Equinix.

Co-location has been a hot-button issue for critics of high frequency trading. I wonder, however, how many have actually visited a co-location facility. In March I got the chance to do just that at Equinix’s 340,000-square-foot NY4 data center in Secaucus, New Jersey. My host was Brandon Travan, head of the information technology infrastructure, co-location and cloud-hosting services divisions for Gravitas Technology, one of the growing number of companies that provide turnkey technology solutions for high frequency traders — and one of the first tenants at NY4.

From the outside, the white, two-story, unmarked building, located 11 miles west of downtown Manhattan in an area known best for outlet shopping, looks like a suburban medical office or law firm. (I later learned that it had been an eyeglass factory.) The lobby is equally nondescript, if not a little odd — there’s not much more than a phone, a plain steel door and a biometric hand scanner. After dialing in a personal code matched against the geometry of his hand, Travan got us into an inner lobby, where three security guards sat behind bulletproof glass and Kevlar-reinforced walls. I gave them my driver’s license (which, I was told, I’d get back when I left), and after two more sets of hand scans and steel doors, we entered the co-location area.

I was glad that I had decided to wear a light coat over my suit that day, because Equinix keeps the facility at a cool 65 to 72 degrees Fahrenheit. The design itself is also very cool. Built on a concrete slab with 45-foot-high ceilings, the building is organized along a rectangular grid system, with rows and rows of servers housed in metal cages for as far as the eye can see. Yellow trays snake above the cages, carrying all types of cables. Orange “innerducts” transport fiber-optic connections within the cages. Giant air-conditioning units, located outside the co-location area in case of a water leak, pump air through large green ducts that come down over the cages, creating a giant convection loop that sends the heat from the servers and networking equipment up toward the ceiling and out through the roof.

The facility is kept dark, both to improve its energy efficiency and to protect the anonymity of Equinix’s tenants. Each cage has lights that come on automatically when someone enters, I learned when Travan typed in the code to unlock his cage. Gravitas has about 35 clients operating out of its space, including one large hedge fund firm that spent more than $1 million on computer hardware, software and setup costs. The majority of Gravitas’s clients, however, are small. The company provides all of its clients with direct high-speed connections to all the market data providers and trade execution networks, including other NY4 tenants, like Direct Edge and the International Securities Exchange.

“We make the electronic trading community of Equinix available to smaller players by taking advantage of economies of scale — helping them get in with the technology they need with almost no up-front capital,” Travan told me. “If you’ve got a coder and the next best algo for trading equities, currencies or other vehicles, instead of needing a quarter-million dollars to start up, you can simply install your code on our hosting platform or send us servers to plug in, and you’re ready to go.”

Not everybody, however, buys the democratization argument. James McCaughan, CEO of Principal Global Investors, says co-location gives high frequency traders an unfair advantage. “Co-location creates an informational asymmetry that is fundamentally acting against the interests of long-term investors,” McCaughan told me exactly one week after my visit to the Equinix data center. “I have no problem with people developing algorithms for high frequency trading as long as they’re doing it with the same information as everybody else.”

McCaughan, whose team manages $215 billion in mostly 401(k) and other retirement assets for Principal Financial Group, considers himself to be a pretty savvy investor. His equity-trading desk has six people at the company’s Des Moines, Iowa, headquarters; two each in London and Singapore; and one in Tokyo, as well as access to state-of-the-art trade-execution algorithms offered by all the leading brokerage firms and third-party vendors. Although there’s nothing stopping Principal from using co-location, McCaughan told me that for a long-term investor, it’s probably not worth the effort. “As a large, sophisticated investor, we can compete,” he said. “But it is a weaker market if you have to be that sophisticated to compete.”

HIGH FREQUENCY TRADING BURST into the public consciousness last summer when news broke that a former Goldman Sachs Group computer programmer had been arrested by Federal Bureau of Investigation agents at Newark Liberty International Airport for allegedly stealing software code. According to the FBI, the programmer, Sergey Aleynikov, transferred thousands of files related to Goldman’s proprietary trading program to his home computers with the intention of using them to help his new employer build a high frequency trading platform. It didn’t take long for that employer, Chicago-based Teza Technologies, to cut its ties with Aleynikov. But Teza’s co-founders soon landed in hot water themselves when just six days after Aleynikov’s arrest, hedge fund firm Citadel sued them for violating a noncompete agreement and trying to steal its trade secrets. Last fall Citadel finally got the injunction against Teza that it was seeking, but by the time the judgment was rendered, the nine-month noncompete period had nearly expired.

The Citadel-Teza lawsuit provides an illuminating window into the world of high frequency trading. The person at the center of the drama is Mikhail Malyshev, a brilliant Russian émigré with a Ph.D. in plasma physics who joined Citadel’s high frequency trading group in 2003. During Malyshev’s six years at Citadel, the firm spent hundreds of millions of dollars researching and developing high frequency trading models and building out the IT infrastructure and systems to implement them. Its market data system, for example, contains roughly 100 times the amount of information in the Library of Congress. Citadel uses this historical data to test its models, which attempt to forecast changes in the prices of securities by analyzing statistical pricing patterns, supply and demand imbalances and other factors. The signals, or alphas, that prove to have predictive power are then translated into computer algorithms, which are integrated into Citadel’s master source code and electronic trading program.

Malyshev oversaw all aspects of Citadel’s nearly 60-person high frequency business, including the approval of trading strategies. He resigned on February 16, 2009; the next day his top lieutenant, Jace Kohlmeier, left too. By April they had incorporated Teza. Last fall, while I was reporting my story on Citadel, Kenneth Griffin, the firm’s billionaire founder and CEO, told me that before the arrest of Aleynikov, he had no idea what Malyshev and Kohlmeier were doing at Teza. After all, he said, the two were still on Citadel’s payroll as part of their noncompete agreements.

Like most hedge fund firms, Citadel is secretive about its investment strategies. The fact that Griffin would pursue a very public lawsuit with Teza’s founders says a lot about the importance of high frequency trading to the $12 billion-in-assets Citadel. In 2008 its high frequency group made $1.15 billion, compared with gains of $892 million the previous year and $75 million in 2005, according to Malyshev’s testimony. The 2008 performance was especially impressive given how poorly Citadel’s large multistrategy funds did that year: Its flagship Kensington and Wellington funds were each down about 55 percent.

Benjamin Blander, who joined the firm in 2004 from Banc One Corp., now leads the high frequency group, which manages a portion of Citadel’s $1.8 billion Tactical Trading fund. Citadel, however, was the only large hedge fund firm I could find that was willing to admit that it does high frequency trading. Most say they use many of the same tools as high frequency traders — employing high-speed computer programs and co-location services to generate, route and execute orders — but that their strategies have a longer time horizon.

“Even the slowest high frequency traders are turning over their portfolios at least a half dozen times a day,” AQR principal Michael Mendelson told me when I met with him in January at the firm’s Greenwich headquarters. “We tend to hold things for at least a day or two, and usually longer.”

Mendelson was head of quantitative equity trading at Goldman Sachs before joining AQR in 2005; he oversees the firm’s statistical arbitrage strategies. He explained to me that high frequency trading doesn’t require much in the way of capital — in fact, it would be hard-pressed to put hundreds of millions of dollars to work. The typical high frequency firm, he added, is likely to have about $5 million in proprietary capital and trade a few million shares a day through a specialized brokerage firm like Lime or Wedbush Securities.

High frequency trading, I learned, is a very low-margin, low-risk strategy. Traders earn less than a penny a share and rarely hold overnight positions. Profits are measured in hundredths of a cent, or “mils,” to use the industry parlance. According to Narang, high frequency traders typically earn about 10 mils, or 0.1 cent, a share trading U.S. equities. One of the attractions of the strategy is its consistency. High frequency traders rarely have losing days. They also tend to do very well during periods of high volatility. May was likely a great month for them.

Narang and other high frequency traders I spoke with gripe about the press, saying that it has often misrepresented what they do and grossly inflated the profitability of their business. They’ve got a legitimate beef. Last summer, a few weeks after the Goldman software-theft news broke, the New York Times ran a front-page story by Charles Duhigg describing how a handful of traders use powerful computers to “reap billions at everyone else’s expense.” The article went on to say that “these systems are so fast they can outsmart or outrun other investors, humans and computers alike,” using flash orders to step in front of those investors. (Under Reg NMS, exchanges were given the ability to “flash” marketable orders electronically for a split second to some professional traders before they are displayed to the broad public.) The article included the bold assertion that high frequency traders generated some $21 billion in profits in 2008.

The source of the data was TABB Group, a New York– and London-based research firm. The problem with the Times story, as I discovered when I met with Larry Tabb at his Wall Street office in early March, was that the $21 billion included a lot more than just high-frequency market making. “That number included anybody following an equities-related time-sensitive strategy that doesn’t take a significant end-of-day position,” the TABB founder and CEO explained. “It is pairs trading, options market making, futures and cash arbitrage, exchange-traded funds.” The profits for the virtual market makers in U.S. equities, he said, were roughly $8 billion in 2008 and $7.2 billion in 2009 and likely to be lower this year because of the drop in volatility and trading volume. (Tradeworx’s Narang says that high frequency trading in U.S. equities generates no more than $2 billion to $4 billion a year in profits overall.)

New York Senator Charles Schumer probably didn’t ask Larry Tabb for clarification after he read the Times article. The very same day it appeared, the longtime lawmaker fired off a letter to Mary ­Schapiro demanding that the SEC do something about high frequency traders and their ability to view order-flow information before the general public by using flash orders. “This kind of unfair access seriously compromises the integrity of our markets and creates a two-tiered system where a privileged group of insiders receives preferential treatment,” he wrote. If the SEC failed to curb flash trades, Schumer threatened to introduce legislation that would.

THE SEC'S NEW HEADQUARTERS IS a marvel of modern architecture. Conveniently located next to Washington’s historic Union Station, the building has a spectacular glass-and-steel atrium where visitors can watch the news on a giant flat-screen television while waiting to make their way into the agency’s inner sanctum. That’s exactly what I was doing one cloudy Tuesday afternoon in late March when John Nester, the SEC’s director of public affairs, came to get me. I’d taken the train down from New York that day to meet with James Brigagliano, deputy director of the agency’s Division of Trading and Markets, and several other members of his team to discuss high frequency trading. The problem was, I didn’t know in advance who was going to be in the meeting, and as Brigagliano and three other staffers from the SEC division filed into the room and quickly introduced themselves, I didn’t catch all of their names. I handed out business cards to each of them in hopes of getting them in return, but only one person (assistant director John Roeser) had brought a card.

It took me about 20 minutes into the interview to piece together the two missing names — “Dan” (who I later found out was market structure counsel Daniel Gray) and “Dave” (associate director David Shillman). Despite my initial confusion, I was impressed with the SEC staff’s knowledge of high frequency trading. We had a wide-ranging discussion, spanning everything from market structure and regulation to Washington politics and financial reform. The SEC’s interest in high frequency trading, I learned, long preceded Schumer’s famous letter to Schapiro last summer (and a similar call from Senator Kaufman for the SEC to do a comprehensive review of market structure). Brigagliano told me that the SEC began hearing about high frequency trading not long after it passed Reg NMS and that his group started working on the equity market structure concept release early last year.

The SEC has a tripartite mission: to protect investors, maintain fair and orderly markets and facilitate capital formation. Although the mandates can sometimes be at odds with one another, getting the first two right can go a long way toward ensuring the third. “We see confidence in the markets as essential for capital formation,” Brigagliano told me. “Investors are not going to commit capital if they think that the system is rigged.”

By the time I met with Brigagliano and his team, the SEC had proposed several new rules to help safeguard the market, including the elimination of flash orders and a prohibition against broker-dealers providing clients with unfiltered, or “naked,” access to exchanges and other alternative trading systems. Naked access has been popular among some speed-conscious high frequency traders because it enables them to place buy and sell orders directly with an exchange or trading venue without being slowed down by pretrade credit and risk checks. The SEC’s rule proposal, if approved, would require brokerage firms to create and maintain strict risk management controls for clients that are given direct sponsored access to electronic trading venues.

Of course, one of the biggest problems the SEC and other regulators have faced is that they simply haven’t had the tools or the data to track the billions and billions of trades that fire across the electronic exchanges and trading platforms each day. Although several of the largest high frequency players, like Getco, are registered broker-dealers and have the reporting requirements that go along with that, the vast majority of firms operate in anonymity. In April the SEC looked to change that when it voted to propose the creation of a large-trader reporting system. If the proposal is approved, any firm or individual that trades 2 million shares or $20 million in exchange-listed securities in a day, or 20 million shares or $200 million in securities in a month, will be required to identify itself to the SEC. The agency will then assign that trader a number, which its broker-dealer will use to tag all its transactions, reporting them, upon request, to the SEC.

“The large-trader reporting system will be simple to put in place,” the SEC’s Shillman told me during my March meeting at the agency. “Creating a consolidated audit trail is more complicated, and could take several years, because it requires systems changes at exchanges and broker-dealers.”

The SEC began working on a consolidated audit-trail proposal last summer, consulting with exchanges and broker-dealers on what would need to be done and how much it would cost. On May 26 the agency unveiled its new rule, which would create a consolidated order-tracking system that would enable the agency to access in real time most of the data needed to reconstruct a market dislocation like the flash crash. But progress doesn’t come cheap: The SEC estimates that it will initially cost about $4 billion to build the system and an additional $2.1 billion a year to maintain it. Taxpayers, however, won’t have to worry about footing the bill; the costs would be borne by broker-dealers, exchanges and other trading venues.

The large-trader tagging and consolidated audit-trail proposals are likely to be approved by the commission over the coming months. Although it’s wishful thinking to expect the SEC to ever be able to match the technological sophistication of high frequency traders, the new rules should eventually give the regulator the necessary tools to monitor and police their activity. I’m sure that is going to make some high frequency traders very nervous, but they shouldn’t be. Unlike some of the more vocal critics of high frequency trading, SEC chairman Schapiro knows that the technological clock cannot be turned back.

“While technology has provided benefits to the market, it has also created real issues,” she told me in early May. “We want to be very careful and thoughtful about how we approach it. The idea that you can say, ‘Let’s just unplug everything’ or ‘Let’s put something into the machines that makes everything go slower’ is probably not realistic.”
In other words, in the battle of man versus machine, both sides could end up winning.