Today Leon Panetta will take the oath to become our nation’s 23rd secretary of defense. After he walks through the Pentagon’s river entrance and up the oak-paneled stairwell to his office, the weight of managing a three million-person department during wartime will sink in. And Mr. Panetta’s tenure begins just as President Obama and bipartisan majorities in Congress are insisting on deep cuts to defense spending. It will be tempting to accede to the White House’s proposal to carve out $400 billion, if not more, from the national security budget by 2023. It would also be a grievous mistake.

Every president and defense secretary inherits the armed forces and military capabilities provided and shaped by their predecessors. Procurement decisions and budgetary actions do not yield results, or reveal their flaws, for years and sometimes decades. With no immediate outward signs of negligence, the political penalties for cutting weapons systems and delaying reinvestment in equipment and infrastructure are close to zero for those in office today. But the penalty for being ill-prepared tomorrow when the unforeseen occurs—whether another terrorist attack at home or a major crisis abroad—can be measured in American lives lost.

The false comfort provided by the end of the Cold War led the administrations of George H.W. Bush and Bill Clinton to draw down the nation’s intelligence and military budgets. As director of the Office of Management and Budget and chief of staff for President Clinton, Mr. Panetta saw that process up close. Its consequences did not become starkly apparent until 9/11, and the resulting scramble to rebuild our capabilities in the months thereafter.

To be sure, there are savings to be found in the defense budget. Mr. Panetta might start by opposing any and all congressional earmarks, past and future. More than $80 billion in unnecessary spending for pet projects has been shoved down the Pentagon’s throat over the last decade.

Mr. Panetta should also reinvigorate the effort to reposition our forces away from unnecessary Cold War defensive garrisons to smaller, forward bases. Until 2005, for example, America was paying $225 million annually to station aircraft in Iceland to monitor threatening Soviet bombers and submarines that hadn’t been around for over a decade. The Bush administration initiated a number of repositioning initiatives in Asia and Europe that have since flagged. The longer we provide a security blanket courtesy of the U.S. taxpayer, the longer we delay the day when European and other allies begin serious investments in their own defense capabilities.

The greatest economies can be found in areas that contain the greatest political peril: military health care and personnel. Defense Secretary Robert Gates has spoken forcefully on the matter, and he created an opportunity for Mr. Panetta to lead. Individual military retirees under 65 currently pay less than $20 in monthly premiums for the Department of Defense’s health plan, called Tricare. The average monthly premium for their private-sector counterparts exceeds $500. Individual contributions will need to increase over time to keep up with spiraling health-care costs.

The U.S. taxpayer is also subsidizing a bloated and broken personnel system for nearly 800,000 Defense Department civilians. Since 9/11, the number of active duty military has increased 4%, while the number of civilians has increased nearly 50%. More recently, the Pentagon decided to increase its acquisition corps to 30,000 from 20,000 civilians. The last thing the nation needs are new and expensive bureaucrats managing fewer programs. The Obama administration should consider a commitment to reduce the DoD civilian work force by 10% through attrition.

While there are substantial savings to be found in the defense budget, hundreds of billions cannot be cut without impairing our security. Mr. Gates has said that he’s already made the “easy” cuts, yet there are serious questions whether some of them—such as reducing the number of F-22 fighters, Navy cruisers, missile-defense interceptors and strategic delivery systems—leave America ill-prepared for a conventional conflict and erode the strong deterrent necessary to prevent it.

Our country has taken an ax to our national security budget—both the Defense Department and the intelligence community—after every war of the 20th century. And every time we later regretted it. After years of grinding conflict, it can be easy to fall prey to the comfortable fiction that the ugly business of conflict is over and that the U.S. can reduce its military and intelligence capabilities. If we revert to the pennywise policies of the 1990s, we are certain to have to once again scramble to rebuild our defenses in the future. The critical difference between today and past eras, however, is that the proliferation of biological, chemical and even nuclear weapons means that America’s margin for error is considerably more modest.

Defense spending is now 19% of federal outlays and declining. This is the lowest percentage since before World War II. At 4.7% of GDP, the defense budget is dwarfed by the cost of Social Security, Medicare and Medicaid, which exceed 10% of GDP. Even if President Obama tomorrow brought home each and every troop in Iraq and Afghanistan, tore down the Pentagon, shuttered the CIA and the national security agencies of government, and pink-slipped the three million men and women defending the country, it would not solve America’s financial woes.

Mr. Gates deserves good marks for his stewardship of the Pentagon, especially for overseeing the surge and the gradual drawdown in Iraq, and helping to drag a hidebound bureaucracy into the 21st century of asymmetric threats. The principal challenge for Mr. Panetta, beyond seeing a successful conclusion to the wars in Iraq, Afghanistan and Libya, will be fending off White House and congressional raids on national security spending.

Mr. Panetta has distinguished himself in his understanding of the exceptional characteristics of American power. What he has seen close up at the CIA he will experience each day in the extraordinary professionalism, capability and will of our fighting forces. The conventional wisdom seems to be that, as a former budget director, Mr. Panetta will know how to skillfully draw down the Pentagon in the “postwar” period to come. We ought to wish him success in proving the conventional wisdom wrong.

Mr. Rumsfeld was secretary of defense from 1975 to 1977 and from 2001 to 2006.