2.0 Program

2.1 Acquisition Strategy

The DARPA High Altitude Unmanned Aerial Vehicle Program (Tier II+) provided the model for the Arsenal Ship acquisition strategy. Tier II+ was the first to use DARPA's Section 845 authority, granted to DARPA in the FY 94 Authorization Act.

Like Tier II+, the Arsenal Ship acquisition strategy was a multi-phase competitive procurement, with each phase reducing the number of competitors and increasing the maturity of the design for both the Demonstrator and production ships. The central element of the strategy was to use competition, with each of the Industry Teams committing to an irrevocable offer for the production ships at the end of Phase II. Phase I's Concept Design was awarded to all qualified full service Teams who could potentially perform all phases of the program. Phase II was to be awarded to two Industry Teams for the Functional Design (similar to Contract Design in maturity). The downselect decision instead awarded three Industry Team agreements based on three very robust Industry designs. Phase III, detail design and construction for the Demonstrator, as well as a priced option for five production ships, was to be awarded to one Industry Team.

Industry was encouraged to form teams to include all of the requisite analysis, design, build capability, and life cycle support required to execute the entire program. The intent was to encourage the creation of a Team where individuals applied their expertise irrespective of company affiliation. This could manifest itself as a joint venture, a limited partnership, or other corporate structure. The classic Prime/Sub relationship was not precluded, but alternative teaming arrangements were suggested as a way to break down corporate stove pipes, as well as achieve efficiencies and save costs.

The use of the Other Transactions Authority eliminated most of the procurement regulations, including those in FAR/DFARS (see Tab E). This allowed a streamlining of the process and shortening of the schedule, especially the time taken to conduct source selection by the Government. It should be noted that even though most of the "rules" are waived by the authority, a review of DoDI 5000.1 showed that the spirit and intent of DoD acquisition policy was being followed. Specific procedures prescribed in DoDI 5000.2 were not emulated.

Concept of Operations (CONOPS) and Ship Capabilities Documents (SCD) were provided as goals (Tabs C and D). The stated philosophy of "no requirements" effectively put responsibility for cost and performance on each Industry Team, and gave them the trade space necessary to achieve PAE goals.

A PAE range of $450M to $550M was established for the average Unit Sailaway Price (USP) of the five production ships. USP and the manning limit of 50 people were treated as the program's only hard requirements.

2.1.2 Lessons Learned

2.1.2.1 Effective competition motivated the Industry Teams to meet the program goals of cost and performance. Clearly the three Phase II Teams felt the competitive pressure of the other consortias and worked hard to create a competitive advantage. It is doubtful that consortia full effectiveness or corporate CEO level participation could have been achieved without the benefit of competition.

2.1.2.2 Industry could effectively form teams containing the requisite capability to perform the contract. The three Teams awarded Phase II agreements created effective organizations that operated mostly as a single unit. However, the members did not lose all of their corporate identity. This was manifest in the degree to which individual company cost data was closely guarded.

2.1.2.3 The shortened source selection process allowed the multi-phase process to work. Historically, the largest issue with pursuing a multi-phase program was the schedule penalty mandated by a lengthy source selection process, often lasting over six months. For Arsenal Ship, slightly less than two months were scheduled for both the Phase II and III downselects, with the selection date cast in concrete and put on all of the principals' schedules months in advance. Government access to Team's designs throughout their development, rather than only at source selection time, was critical to being able to execute the evaluation on a short schedule. The climate of rapid prototyping within DARPA, the willingness to explore new concepts, and the focused support by Director, DARPA, greatly facilitated this process.

2.1.2.4 One of the advantages of using the Agreements Authority was deleting "protest avoidance" from the process. Many of the rules established by the acquisition community over the years to avoid litigation have become cumbersome and detrimental to an efficient process. Although the integrity of the individual Arsenal Ship Team design and business data were jealously guarded, the "fairness" rules¹, for example, were eliminated, a fact which greatly improved the ability of the Government to support and communicate with the individual Teams.

2.1.2.5 When cost is the only requirement, it is essential that both the development budget and PAE goal be achievable and balanced. The $450M PAE goal for average production USP turned out to be a good number and universally supported by industry. Industry's limit of $389M for development of the demonstrator ship was extremely challenging as compared to the average production cost goal of $450M. Industry pushed some software development to the production ship to reduce demonstrator ship non-recurring costs, with attendant reduction in demonstrator capabilities and increased production risks.

2.1.2.6 A streamlined acquisition process can significantly reduce the time and cost of bringing the product to market. The process being followed by Arsenal Ship demonstrated a 50% reduction in acquisition time for the design portion of the ship compared to the traditional design approach. The Industry Teams were prepared to complete the detail design and build the ship in 33 months, again about half the normal time. This was primarily enabled by using an industry led acquisition operating under Section 845 authority, with industry having full trade space and responsibility for the design.

2.1.2.7 One power of Section 845 Authority was in changing the mindset of the participants. The acquisition process currently in place incentivizes risk averse behavior, and stifles innovation. True acquisition reform and streamlining is possible if all the rules are taken away first and then reinserted on a case basis, such as for security and explosive safety. If each exception to the conventional rules must be justified, streamlining will not occur. Section 845 lets good business decisions govern.

2.1.2.8 Developing a long term vision and fully disclosing it to industry at the outset sets the stage for the entire program. In order for industry to plan for and invest in a program it has to know the Government's plans. Industry can seek this through the back door or Government can put the word out openly and consistently. Arsenal Ship sketched the program vision in two industry day sessions early, and then reinforced the vision themes at each industry interaction.

2.1.2.9 Allowing industry to set requirements increases design innovation. By not invoking any requirements, the Government gave Industry the total trade space for the overall design and the subsystems selection. Consequently, the use of new technologies integrated in different ways was facilitated, especially in the areas of communication, information systems and topside integration. Opening the development filters permitted the next logical step from the Goldwater Nichols Act of 1986; joint warfighting design at program inception.

2.2 Contracts (Agreements) and Legal

2.2.1.1 Key Events

Phase I
Draft Solicitation and Industry Briefing	7 May 96
Industry Discussions	14 May 96
Final Phase I Solicitation	23 May 96
Phase I Proposals Received	25 June 96
Phase I Selections Announced	11 July 96
Negotiations Concluded (Agreements executed - five Teams)	Aug 96
Phase II
Draft Solicitation	13 Sept 96
Final Phase II Solicitation	3 Oct 96
Phase II Proposals Received	15 Nov 96
Phase II Selections Announced	10 Jan 97
Negotiations Concluded (Agreement mods executed (3 Teams)	Feb 97
Phase III
1st Draft Solicitation	16 Jun 97
Draft Phase III Model Agreement Modification	15 Jul 97
Commence Phase III Agreement Mod negotiations	Aug 97
2nd Draft Solicitation	1 Aug 97
Final Phase III Solicitation	15 Sept 97
Cancellation of Program/Solicitation	30 Oct 97
Phase III Proposals Due	14 Nov 97
Phase III Award Scheduled	16 Jan 98

2.2.1.2 Approach The contracting approach for the Arsenal Ship program was to utilize P.L. 103-160 Section 845, Other Transactions Authority. This authority was provided to DARPA by the Congress in FY94 and has since been passed on to the military services by PL 104-201 Section 804. DARPA had experience with using 845 authority on other R&D prototyping projects. The Arsenal Ship Program's acquisition strategy was modeled after DARPA's Tier II+ program for Unmanned Aerial Vehicles. Use of Section 845 Authority allows for the procurement of prototypes outside FAR and DFARS regulations. Additionally, the Arsenal Ship Program was designated as a "non-ACAT" program (Tab A) , relieving it from the requirements of DoDI 5000.2. The Contracting Officer was free to negotiate terms and conditions specific to the individual needs of the program.

There were several motivating factors that support use of Section 845, including: 1) enticing non-traditional DoD companies, at both the prime and subcontract level, into the business; 2) facilitating insertion of commercial technology; 3) encouraging innovation, and; 4) decreasing the overall time and cost to design, build and deliver a ship. The solicitations for each Phase of the program contained a "model agreement" with notional terms and conditions appropriate for that particular phase. Industry Teams were free to propose their own terms and conditions to the agreement based upon their particular circumstances. Agreements were then negotiated with each Team individually.

2.2.2 Lessons Learned

2.2.2.1 Non-traditional DoD companies are willing to do business with the Government using Section 845 Other Transactions. During Phase I of the program, one non-traditional company participated at the prime contract level. They proposed the use of a unique construction technique to which it owns patents. Use of Section 845 definitely provided an avenue for this Team to propose use of its proprietary technology because FAR/DFARS rights were not invoked during the competitive process. Additionally, non-traditional DoD companies also participated at the subcontractor level. Hopefully, this will lead to new business relations between traditional and non-traditional DoD companies and facilitate the introduction of the latest commercial technologies into Government acquisition.

2.2.2.2 Industry retention of most data rights under Section 845 facilitated innovation. Section 845 allowed for modified data rights which encouraged innovation during competitive phases. Industry Teams were allowed to retain data rights in the event they were unsuccessful during any downselect process. Teams were eager to put forth their best efforts without the fear that their innovations would be taken by the Government and distributed to other Teams. The Government clearly stated its intent with regard to data rights at the onset of the program - i.e., should the Team be chosen to construct the MFSD/Arsenal Ships, the Government wants rights to all data necessary only to maintain, modernize and support the ships. Rights to data would be defined by what the data is used for, instead of who paid for or prepared the data, as is the case under FAR/DFARS data rights clauses. Defining rights by their usage, as opposed to whether the Government paid for the rights, was believed to be a better approach to facilitating proper use of the data.

2.2.2.3 The intent to eliminate Government Furnished Equipment (GFE) facilitated innovation during the competitive phases of the program and could dramatically reduce the Government's risk of claims. No GFE was envisioned under the Arsenal Ship Program. This allowed for maximum design flexibility and took the Government out of its GFE responsibility role. This approach allowed for extensive review of new ways to satisfy the Government's technical requirements while potentially reducing production, maintenance and installation costs. Elimination of GFE placed the Industry Team in charge of the final product and greatly reduces opportunities for claims related to poor specifications, late GFE, inoperable GFE, etc. Some Teams discussed the potential use of GFE for particular systems if it appeared that it would be more economical for the Government to purchase a system through a quantity buy. The Arsenal Ship program was concluded before the full implications of such an approach could be explored or negotiated.

2.2.2.4 Access to data created under traditional Government contracts for legacy systems is extremely difficult. Because of the Arsenal Ship's operational interdependency on Government legacy systems already in the fleet, access to data on these legacy systems was necessary in order for industry to perform design trades and engineering development. Access to data on Government legacy systems proved to be extremely difficult and time consuming despite extensive efforts by the Industry Teams and the ASJPO. The experience indicates that despite data rights provided by the FAR/DFARS, Government offices either questioned the Team's rights/ability to utilize any of Government's data or simply refused to provide it to other Government agencies for use on their programs. As a result, industry had to make assumptions regarding legacy system operations or even in some cases reverse engineer aspects of their design to ensure compatibility with legacy systems. Such action is time consuming, inefficient, potentially costly and underscores that traditional data rights clauses are counter productive.

2.2.2.5 The implications of an "irrevocable offer", which fixed the cost of production units, provided tremendous leverage to the Government and incentive to the Industry Teams to design to cost. At the conclusion of Phase II, all competing Teams were to provide an irrevocable offer to build five production Arsenal Ships at a fixed price. The detailed design of this Arsenal Ship would be defined during the Phase III construction of the MFS Demonstrator. Industry Teams were driven to propose the best capability for the money as a result of competition. However, they were also driven to develop a design that was realistic for the cost as a result of the irrevocable offer. If the design proved too expensive, the Industry Team would lose money on the production ships. However, if the design lacked in capability, the Industry Team ran the risk of not being continued in the program. These dynamics were created to facilitate true PAE design development and credibility in cost estimating. All Industry Teams viewed the Irrevocable Offer seriously from a contractual stand-point. The Irrevocable Offer drove Industry Teams to develop the most realistic, vice optimistic, cost estimates because they were contractually committed for these estimates at a fixed price. Negotiations of contractual language pertaining to the Irrevocable Offer were extremely complex and time consuming.

2.2.2.6 Negotiation of Section 845 Agreement terms and conditions facilitates greater understanding between the parties. Use of Section 845 Other Agreements Authority, provides both Government and Industry with the opportunity to deviate from traditional DoD terms and conditions when it is appropriate for the circumstances. With such opportunity comes a great responsibility to fully understand all technical and business aspects from which the Industry Team is operating. It allows both sides to analyze why a particular procurement can or cannot be performed under traditional terms and make exceptions to such terms if it will facilitate development under the program that will ultimately benefit both the Government and Industry. As a result, negotiations may become very complex and require understanding of industry's motives and concerns in order to determine what is in the best interest of the Government. It is extremely important that legal and technical personnel, as well as contracting personnel are actively involved in the negotiation process.

2.2.2.7 Effective implementation of Section 845 authority required close teamwork among the Government's contracting, legal and technical team members. The lack of established rules or regulatory guidance inherent in the use of Section 845 authority necessitated close communication between the Government contracting, legal and technical team members. Two major attributes contributed to this successful working relationship. First, the small size of the program office allowed for rapid access to the necessary team members, especially the program manager, for discussion and decision. Second, the team members in the ASJPO were specifically selected for their ability to be open-minded, flexible and willing to experiment with the authority granted under Section 845. These attributes were particularly important in the contracting and legal team members, whose personnel have been traditionally viewed as impediments. The support provided by DARPA General Counsel and DARPA Contracts Management Office, was invaluable.

2.3 Role of Government

2.3.1.1 ASJPO Relationship to Industry Teams

ASJPO objectives differed by program phase. During Phase I, the strategy for engagement was guided by the objective to permit uninhibited concept exploration and innovation by Industry Teams. To this end, Government involvement was structured as "objective" in nature, that is, data or calculation results were made available to the Teams but no advice of any kind was provided. The resulting industry-led efforts produced the desired significant innovation as characterized by:

• Major system innovations
• Independent, highly competitive cost estimates
• Introduction of commercial practices
• Aggressive schedules
• Effective teaming of systems integrators and shipbuilders

The objective for Phase II was to ensure successful functional design development that would result in highly desirable competitive proposals for construction. This objective demanded a more pro-active involvement to ensure the development of a ship that the fleet would evaluate positively in all areas. This approach could be taken without jeopardizing the Phase I success because:

• Remaining trade-offs were primarily at the subsystem level with limited cost impact
• Operational (user) input is more important as design detail unfolds
• Government had specific areas of expertise not generally available from industry (e.g., survivability)
• Under Section 845 Agreements, industry was not required to take Government advice

• Interact in a supportive way on a minimally invasive basis during reviews, technical interchange meetings, and internal team meetings. Coach each Team to be the best they can be within the context of their approach.
• Encourage consideration of out-of-the-box opportunities for technology insertion and business practice reform. Provide real-time feedback.
• Facilitate access for Teams to Government offices, information and software.
• Continuous involvement of ASJPO personnel in the design reviews and open discussions increased understanding of the approach, allowed for identification, disclosure and correction of major deficiencies and shortened the evaluation process following formal proposal submission. It also increased productivity on both sides.

The Industry Team members faced a similar cultural shift. It took some initial convincing on the part of the ASJPO before the Industry Teams believed that the program office was serious in pursuing a more flexible and commercial-like way of doing business.

During Phase III it was anticipated that a very close working relationship with the single winning Industry Team would evolve, including on-site ASJPO representatives who would answer questions and facilitate getting Government data, coordinating program functions with other Government activities (such as WSESRB, fleet operational testing, INFOSEC security, testing ranges, etc), and perhaps even participate directly in software development via the Government R&D Centers. However, the responsibility for defining the best product for the fleet would remain with the Industry Team.

2.3.1.2 ASJPO relationship to support Contractors and Government R&D Centers

ASJPO recognized prior to the initiation of Phase I that specific expertise in analysis of ship motions, survivability, signature measurement, C4ISR, Life Cycle Management and ship production was needed to augment the Program Office. Contractual arrangements were negotiated with several private companies including: SYNTEK, Vail, SPC, and LMI. Additionally, task statements were put in place at NSWC Carderock, NSWC Dahlgren (to include Coastal Systems Station), NAWC China Lake (to include Point Mugu), NSWC Port Hueneme Division East Coast Operation, NCCOSC NRaD, JHU/APL, and NRL, to provide support to the ASJPO. These particular private industry Government Laboratory personnel were identified as source selection assistants, and were barred from interacting with Industry IPTs. All personnel included in internal ASJPO activities submitted Financial Disclosure Statements which were screened and approved by DARPA General Counsel for potential conflict of interest.

2.3.1.3 Government R&D Centers relationship to Industry Teams

ASJPO made available to the Industry Teams, government services that could not be obtained from private industry. A Phase I Memorandum of Agreement (MOA) (Tab K) was negotiated between NSWC and ASJPO that outlined the allowable interaction with industry during the first phase of the program. This allowed the Laboratories to provide objective products and services, non exclusively.

In Phase II, the relationship between the Industry Team and the Laboratories was expanded (Tab L) to include lab participation on Industry IPTs. Great flexibility was provided by the MOA; however, because of R&D Center concern with the legal restriction that they not compete with private industry, R&D Center personnel were slow to start work on industry IPTs. To alleviate this problem during Phase II, funding ($450k per Team) was made available at the R&D Centers directly from ASJPO to support each Industry Team. Industry Teams developed Statements of Work for this support since the Government itself (ASJPO) was providing the funds, the R&D Centers felt much more at ease with industry requests. This arrangement avoided any competition with industry and worked extremely well.

2.3.1.4 ASJPO and Industry Team relationship to PARMs

Within the trade space available to the Industry Teams to satisfy Arsenal Ship design goals, was the opportunity to select legacy Navy systems. The use of legacy systems in many cases was more attractive than developing new ones based on investment required, and because these systems have passed the WSESRB, OPTEVFOR and other safety/operational checkpoints in other developmental programs. These systems could be obtained either as Contractor Furnished Equipment (CFE), or purchased through the Navy PARMS. PARMs (PARticipating Managers) are the Government program offices responsible for individual Navy systems such as ATWCS, CEC, and SSDS. Systems of primary interest to Arsenal Ship Teams were ATWCS, AFATDS, and CEC.

The PARMs were receptive to making hardware available to the Arsenal Ship Teams, but expressed concerns about requests for the release of source code software to Industry Teams. They cited the difficulty in maintaining configuration control, plus the apparent liability of the PARM for performance anomalies in his system but was not under its control. As a consequence, PARMs were unwilling to release Government owned software for Arsenal Ship program use.

These concerns were the subject of an Arsenal Ship Offboard Systems Working Group, discussed in the C4I section of this report. Although not resolved during Phase II, it was hoped that a good working relationship could be developed during Phase III when only a single Industry Team would have interest in modifying source code. Initiatives in ATHENA working groups would likely have facilitated this process.

2.3.2 Lessons Learned

2.3.2.1 A small Government program office is sufficient when the total system design responsibility is in industry's hands. The six Government personnel and six technical SETA proved adequate to monitor the consortias' progress, without imposing excessive oversight. By maintaining a daily consortia calendar across the teams, ASJPO could be physically present at the majority of design interactions by the teams. By locating access to their Integrated Product Design Environments (IPDE) at ASJPO, real time accessing of trade studies, IMS, IMP, 2D and 3D CAD/CAM models permitted virtual presence for those periods when ASJPO could not be physically present.

2.3.2.2 Industry welcomes openness of Government feedback. Industry was solicitous of Government feedback, to avoid revisiting errors of previous design efforts (which cost time and money), and to hear directly from the customer on likes/dislikes. They welcomed the openness of ASJPO comments on their progress. This relationship was enabled by two aspects of the Agreement; no claim provisions and no unilateral Government direction.

2.3.2.3 Providing expertise and "lessons learned" to Industry is a new Government role. Much as it was difficult for industry to accept full design responsibility at the beginning of Phase I, it was equally difficult for the Government to provide advice rather than give direction. Government retains a wealth of knowledge and operational experience that is indispensable to industry's design/build process. ASJPO called on technical specialists to provide briefings on survivability, combat systems interfaces, and Joint warfighting C4ISR. Access to intelligence reports, studies, and government designs were also facilitated by ASJPO. The guidance to Government personnel was that information exchange was acceptable, but that design mandates were not.

2.3.2.4 Government intervention was required in obtaining data, access, and equipment. Industry does not have authority to compel PARMs or System Command headquarters to provide information and access to data or equipment. ASJPO frequently had to exercise such authority.

2.3.2.5 Safety and security cannot be delegated to industry. Industry's responsibility for and integration of all aspects of the design/build product requires a special interaction with the Government Program Office in the case of safety and security. The Program Office is ultimately responsible for personnel and ordnance safety. Extant Government agencies (WSESRB and INFOSEC, for example) do not respond to private industry, but are chartered to respond to the Program Manager. In this special relationship, the Government must fully understand the design, and certify its safety.

2.3.2.6 Direct funding of R&D Centers by the Program Office is essential to performance of some tasks. Due to legal restrictions on Government R&D Centers "not to compete" for services available from private industry, direct Government funding is needed to provide flexibility for some industry tasking. For example, combat system engineering is available from a variety of industry sources; however, for a specific system, this may not be true. In this case, the Government must identify work-arounds to facilitate competition.

2.3.2.7 The Government team should have expertise, insight into, and understanding of Teams' business decisions as well as technical decisions. In some cases the Industry Teams made design decisions because they were either the developer of a system/software, because they could not get detailed information about a system/software developed by others, or because their real market for a particular system is another program. The Government team needs to understand the motivations which underlie specific design decisions in order to assess if those design trades yield the best solution.

2.4 Funding

The funding for the Arsenal Ship program was initially laid out in the Memorandum of Agreement (MOA) for the Joint Navy/DARPA Arsenal Ship Demonstration Program between Mr. Larry Lynn (Director, DARPA), Mr. John Douglass (ASN(RDA)), and RADM D. J. Murphy, USN (N86) in May 1996 (see Tab B). The cost of the R&D program for the Arsenal Ship Demonstrator was determined to not exceed $520 million including the cost of concept development and competition. The Navy DARPA joint funding profile was:

	FY96	FY97	FY98	FY99	FY00	FY01	Total
Navy	$4.0	$25.0	$141.0	$90.0	$80.0	$10.0	$350.0
DARPA	$1.0	$15.0	$47.0	$50.0	$36.0	$21.0	$170.0

In the same MOA the Navy agreed to provide its share of the funds to DARPA at the beginning of each fiscal year.

In January 1997, during the Phase I to Phase II downselect, three team's designs were determined to be of such value as to warrant their continuation into Phase II. The cost of adding a third Industry Team to Phase II was calculated to be $21M which the Navy agreed to fund. The $21M was split funded between FY97 ($9M), and FY98 ($12M).

During the PR-99 review process, the Navy redistributed $50M from FY98 to FY99; the overall Navy total of $350M remained the same. A dialog with industry determined that this revised funding stream could be accommodated without impacting the delivery and test schedule of the Demonstrator.

2.4.2 Lessons Learned

2.4.2.1 Funding Navy Research and Development (R&D) Centers/University Affiliated Research Centers (UARC) to Support Industry was necessary. Money sent to R&D Centers from ASJPO on behalf of the Industry Teams was necessary to maintain each Team's competitive edge and to allow for specific support. R&D Centers were allowed to perform any kind of support work without regard to "non-competitive" rules of accepting money directly from industry.

2.4.2.2 The ability of a University Affiliated Research Center (UARC) to support Industry needs to be broadened. Industry relationships with UARCs are much more difficult to establish than those Government Laboratories with DBOF funding. UARC's, much more than DBOF laboratories have been concerned with maintaining their appearance of objectivity with its many customers. The legal barriers of industry UARC interaction need to be more fully explored in the future to enable a much needed dialog.

2.5 Source Selection

The Arsenal Ship Program utilized "best value" in determining the successful offerors for each phase of the program. A new solicitation was released for each competitive phase of the program that contained the source selection criteria applicable to that particular phase. The entire solicitation, including source selection criteria, was released to industry in draft form to allow them an opportunity to provide input and ensure clarity.

Source Selections for all Phases of the Arsenal Ship Program were performed in accordance with the approved Source Selection Plans. A new plan was prepared and approved for each consecutive phase of the Arsenal Ship Program. The Source Selection Plans were the same in structure and format to those prepared under traditional Government source selections. All source selection criteria and any information regarding its interpretation were contained in the solicitation. The source selection plan did not provide any elaboration of the selection criteria beyond that which was shared with industry in the solicitation.

The Arsenal Ship source selection approach was to perform an integrated technical, financial and management review of each Team's proposal. All source selection participants were required to attend one of several briefing sessions that outlined the source selection process, criteria and schedule. Source selection participants were divided into teams based upon their area of expertise. However, there was cross-over of personnel on all teams (i.e., some technical team members were also cost or management team members, etc.). The contracting, legal and technical team members all read and commented on all parts of the proposals and all participated in the discussions and selection of the ultimate awardees. Each team had a designated team leader who coordinated the team's evaluation efforts. Team leaders met briefly each morning to address questions or issues that may have affected the other teams' efforts and ensured a free exchange of ideas and information across the technical, financial and management disciplines.

In addition to the written proposal, oral presentations were held with industry approximately three weeks after receipt of proposals. Dates for oral presentations were assigned to each Industry Team by random drawing. All Teams were provided equal time to present highlights to their proposal. Oral presentations provided the ASJPO with the opportunity to clarify aspects of each Team's proposal. The solicitation specifically stated that oral presentations were not an opportunity to revise proposals and that the Government did not intend to request revised proposals as a result of the oral presentations. All oral presentations were held at the Team's facility and were video taped, with a copy to the Government. Only ASJPO personnel attended these oral presentations.

Evaluations from all source selection participants were consolidated into team briefings that were presented to the Source Selection Evaluation Board (SSEB). The SSEB then provided these team briefings to the Source Selection Advisory Committee (SSAC), along with its recommendations for award, over a period of two days. The SSAC was provided an opportunity to comment and question the results of the SSEB. A consolidated briefing, including the comments and recommendations of the SSAC, was then delivered by the SSEB to the Source Selection Authority (SSA). The SSA made his decision based upon this briefing within a day.

After the award announcement, detailed debriefings were held with both successful and unsuccessful offerors covering cost, management and technical issues.

2.5.2 Lessons Learned

2.5.2.1 Detailed debriefs to successful offerors provided Teams full insight into perceived strengths & weaknesses which allowed them to improve their efforts in subsequent phases and reduce risk for the program. The ASJPO provided detailed debriefs to all offerors, including successful offerors. The source selection brief given to the SSA was used as the basis for detailing strengths and weaknesses in the Teams' proposals. The detailed debriefs to the successful offerors proved extremely beneficial at enlightening the Teams regarding areas where the Government perceived excessive technical risk, questioned costs or the management approach. All Teams made an extensive effort at redressing questionable areas noted in the debriefs. This led to improved designs and overall risk reduction for the program. Industry commented that they felt these debriefs were extremely beneficial, and the best they had received in twenty years.

2.5.2.2 Cross-over of source selection personnel (i.e., technical, financial and management) is imperative to evaluating proposals. The Arsenal Ship Program allowed industry to develop its own ship design. Because each Team's design, management and production approach differed, cost estimating had to be very individualized. The financial analysis team's results were very dependent on the findings of the technical team with regard to the design being proposed and the production methods utilized. Likewise, the management team's results were important to both the technical and financial teams' analysis. Insight into the findings of the financial and management teams provided the technical team further understanding of the proposal and potential risks. All source selection personnel were provided full access to all parts of the Industry Teams' proposals, including cost data. All source selection participants agreed that this full and open approach led to a greater understanding of what was being proposed, eliminated many questions and guessing, allowed for better identification of risks, and facilitated a true best-value decision.

2.5.2.3 Continuous interaction with Industry Teams throughout the design phases facilitated a short source selection schedule. The Phase I source selection for the Arsenal Ship Program was completed within three weeks from the time of receipt of proposals to the announcement of successful offerors. Phase I source selection criteria's main focus was that offerors assemble a team fully capable of performing all aspects of the Arsenal Ship Program (i.e., trade-off analysis, design, construction, and support). Six proposals were received and five agreements were awarded. However, the Phase II source selection criteria required detailed analysis of each Team's concept design in order to determine which represented best value to the Government. The Phase II source selection was completed within two months (Nov 15-Jan 11) from the time of receipt of proposals to the announcement of successful offerors. Given the large amount of information presented and the detailed analysis required, this schedule presented a great challenge. The ASJPO believes this schedule was made possible due to the intense amount of interaction that occurred between the Teams and the ASJPO while industry was developing their concept design. Technical, cost and management approaches were continuously discussed prior to and during the Phase II and III periods of performance. As such, the ASJPO was provided great insight into what the Team's proposal would contain. Risks were continuously assessed and discussed. Proposals presented by the Teams built upon work that had been presented throughout the year.

2.5.2.4 The ASJPO's ability to strictly adhere to schedules established at the onset of the program greatly enhanced program credibility and the commitment of the Industry Teams. In the Phase I Arsenal Ship Program Solicitation, the Government provided a top level program schedule addressing all phases of the program. This schedule indicated start/end dates for each phase and indicated release dates for solicitations, due dates for proposals and announcement dates of successful offerors for each phase. Every program milestone was met as originally scheduled for the eighteen months of Phases I and II. The ASJPO received tremendous feedback from the Industry Teams regarding its ability to establish a schedule up front and adhere to it. The Government's ability to manage its own portions of the program schedule facilitated the Industry Teams' ability to plan their own program schedule with Team members and subcontractors and execute accordingly without fear of delay on the part of the Government, which is all too typical in the majority of Government programs.

2.6 Testing and Certification

Early in the Arsenal Ship program, the program office decided to pursue substitute methods for the traditional processes of test and evaluation and ship acceptance. As a consequence, provisions were included in the Phase II solicitation that required the Industry Teams to develop processes and procedures by which the Arsenal Ship Demonstrator and the Production Arsenal Ship would be certified as "fit and safe for the intended service." The purpose of this part of the Arsenal Ship "experiment" was to define and implement a streamlined process that would achieve the intent of Navy T&E and INSURV at less cost and in less time.

The Arsenal Ship Program Agreement required the Industry Team to provide the Government a Preliminary Certification at the end of Phase III. The Preliminary Certification signaled the end of Phase III and the readiness of the Demonstrator to proceed into Phase IV, Demonstration. A Final Certification report was also required of the Team. This report was to detail the test and evaluation results of the Phase IV Demonstration; define the capabilities of the Demonstrator, and verify the military utility of the Arsenal Ship. Receipt of the Final Certification Report signified the completion of Phase IV. The Government would accept the Demonstrator upon receipt of the final report or upon completion of deficiencies whichever was later.

ASJPO placed responsibility on Industry to identify and analyze the constituent elements of the current Navy test & evaluation and acceptance process and find acceptable alternatives that reduce time and cost. One cost reduction strategy was to reduce redundancy in tests and inspections. Self certification, as an example, is one way to do that. By the end of Phase II each Team had developed a plan and a proposed program by which they would "certify" both the Demonstrator and the Production Arsenal Ships.

2.6.2 Lessons Learned

2.6.2.1 An alternative to the traditional T&E and Acceptance process is possible as part of ship acquisition and that process can result in less cost and time without compromising safety, reliability, or performance. The work completed by the three Industry Teams, including preparation of a Demonstrator Test Plan, and a plan for Certification by the end of Phase II work, supports this conclusion. Inherent in the meaning of "certification" as "fit and safe for intended service" is a process by which all tests, analyses, inspections, etc. essential to verification that specific performance and safety requirements are met, are identified and completed. This then requires the Industry Team to establish traceability among all performance/safety requirements and the certification events necessary for their verification.

2.6.2.2 Some certifications are best performed by Government bodies. Certain certifications currently performed by Government activities, e.g., weapons safety by WSESRB, were deemed essential and had no acceptable alternatives. Other certifications, e.g., that for helicopters, were reviewed with an eye to change in order to improve performance. In the latter case, the rule makers were willing to make changes as long as safety principles were retained. There was mutual agreement between ASJPO and the Industry Teams that these special cases would be incorporated into each Team's certification plan.

2.6.2.3 Certain unique range and test capabilities are owned by the Government (e.g., Eglin and Point Mugu missile test ranges) and it makes no sense to invest in any new facilities or replicate existing capabilities. DoD policy is supportive of industry use of Government ranges. Industry Team interaction and discussions with managers of various Government test facilities were positive and indicated that industry could avail themselves of these facilities in cooperative arrangements with Government. At facilities that are not unique, range managers will be challenged to control costs in order to remain competitive.

2.6.2.4 Classification Societies represent a resource to provide some cost effective inspections, analyses and commercial certifications. Classification Societies such as the American Bureau of Shipping (ABS) can perform certain hull and mechanical system certifications but do not have the background, experience nor the personnel to address uniquely military systems such as combat systems. ABS has developed a proposed Certification plan, which identifies by SWBS group, systems to be certified, appropriate standards to be applied, and recommended certification agents. The ABS work lends confidence to a movement towards commercial certifications.

2.6.2.5 Early involvement by Navy T&E activities in Industry-led IPTs is desirable. Currently, operational ship test planning expertise and knowledge resides in the Government and most range testing done for ships has been coordinated by military and Government civilians; therefore, these individuals can bring a wealth of information to the design and development process. This has to be controlled by the program office to insure that inefficient methods and procedures are not perpetuated and that the Government interaction has value added to the Industry Teams. The role of COMOPTEVFOR in demonstrations designed to support production decisions should be determined early in the program. OPTEVFOR participation is not essential to execute an effective program; however, their participation in Industry Team-led IPTs will bring operational credibility and experience to the process.

2.6.2.6 The issue of Live Fire Test (LFT&E) requirements and how they are handled within an Industry Team-led environment needs to be addressed. Arsenal Ship as a non-ACAT program was a special case not bound by LFT&E legislation. Other programs subject to this legislation will have to address the best mechanism to satisfy this requirement. In particular, some of the current requirements to perform vulnerability assessments, which could dictate design choices, may run counter to unconstrained performance trade space.

2.6.2.7 Some Government furnished items may be appropriate in the context of operational test and evaluation and demonstrations. Future ship acquisition programs will likely strive to minimize Government furnished equipment(GFE); however, the Navy should supply military or other unique, Government-owned items directly to the Industry Team if that is cost effective and facilitates meeting schedule milestones. Examples include missiles and other ordnance for demonstrations that may not be easily obtained by an Industry Team.

2.6.2.8 Ship Designation, "public" or "private," determines the applicability of U.S. Coast Guard and other international regulations. An early determination needs to be made whether a ship is a public or private vessel. Private vessels must comply with U.S.C.G. regulations. 46 USCA 2109 exempts public vessels from these same regulations. U.S. Coast Guard jurisdiction is determined by ownership and ownership in turn implies indemnification and liability. For Navy Ships, DoD (Navy) initiates a waiver of compliance for all life saving, navigation, and other regulations. Regulations for carrying/packaging hazardous materials are handled by a different organization within the Department of Transportation called the Research and Special Projects Administration (RESPA).

2.6.2.9 Early establishment of Industry T&E IPTs and Government participation in them is important. Active Government participation in Arsenal Ship T&E IPTs proved to be very beneficial and an effective way to introduce Government input and thinking into the Industry test and certification process. Early involvement of T&E personnel on other IPTs and in the design development process is essential, particularly when testing is part of a risk reduction program. Risk reduction tests and demonstrations need to be tracked and monitored at the Program office level.

2.6.2.10 User participation and Fleet Exercise planning in Industry tests and demonstrations should be coordinated by the program office. The Government program office should arrange for the availability and participation of active duty military and Government civilian personnel to support T&E and demonstration activities. The program office should also coordinate ship participation in fleet exercises.

2.7 Office Operations

2.7.1.1 Organization

A small program office was an essential element of the ASJPO acquisition reform approach. A large number of people would have had a stifling affect on industry's innovation just from the sheer magnitude of opinions that would be expressed by such a group and industry's intense desire to please its customers. The make up of the office dictated by the original Memorandum of Agreement of 28 May 1996 among DARPA, ASN(RDA) and N86 specified six Government employees, growing to a maximum of 9 during Phase III. DARPA, NAVSEA, and ONR were to "contribute" (long term loan) 3 persons each, thus no billets beyond that for the Program Manager at DARPA were created for the program office.

Additional private industry personnel were collocated in the Program Office and dedicated full time. The companies, Vail Research and Technology Corporation and SYNTEK Inc., were barred from any contractual relationship with the competing Arsenal Ship Industry Teams. This approach is very much in keeping with past DARPA program office structures.

The total on-site staff by early Phase II (spring 1997) was as follows:

Program Manager	Capt. Charles Hamilton	DARPA
Contracts Negotiator	Cindy Shaver	NAVSEA (02)
Technical Director	Kit Ryan	NAVSEA (03)
C4I Engineer	Jack Frink	ONR (IPA from JHU/APL)
Combat Systems Eng.	Terry Sheehan	NSWC/Dahlgren
Survivability Engineer	John Grizzard	ONR
Admin assistance	8 people	Vail Research and Technology Corp.
Technical assistance	6 people	SYNTEK
	__________
Total on-site	20 people

Significant off-site staff support was provided by:

General Counsel	Rick Dunn	DARPA
Assistant General Counsel	Dianne Sidebottom	DARPA
Contracts Management Directorate	John Ablard	DARPA

Support personnel were fully representative of the program office to the Industry Teams and to other Government activities; close integration of the support contractor team with the Government Program Office while avoiding personal services and inherently governmental function issues proved possible through open and continuous communication and clear understanding of roles. The support team consisted of:

Hull, Mechanical Electrical Engineer	Robert Percival	SYNTEK
Ship Operations and Support Eng.	Dr. Bob Johnson	SYNTEK
Systems Engineer	Ron Kiss	SYNTEK
Test and Certification Engineer	Jack Turner	SYNTEK
Operational Effectiveness Engineer	Bruce Dyer	SYNTEK
C4I Supporting Engineer	Mike Yermakov	SYNTEK
Ship Cost Engineer	Dave Schwiering	Vail
Office Manager	Jennifer Bennett	Vail
Financial Manager	Jennifer Sowa	Vail
Security Manager	Jim Booth	Vail
Computer Manager	Jack Stevens	Vail
Graphic Artist	Jill Judge	Vail
Administrative/Contracts Assistant	Wendy Turner	Vail
Travel Assistant/Receptionist	Frances Horner	Vail

2.7.1.2 Information and document control

On-site capability for handling documents through the Secret level was maintained. Front door access required a special access badge issued by DARPA. Only the above on-site personnel were issued these badges.

An acknowledged special access compartment was established to handle compartmented data at a site close by which was already functioning for this purpose. This greatly facilitated handling such data, discussions and even travel arrangements.

A classification guide was developed with the objective of making design information easily available to designers and production people for the life of the program. Thus, ship lines were made FOUO, for example. Even so, some of the Industry Team personnel were inadequately cleared in advance for sensitive data, which hindered Team communication and, sometimes, decision making. Structural design and signature analysis (and their effect on ship systems engineering) suffered from some of these restrictions on personnel.

2.7.1.3 Communication via IPDE

The goal of keeping in close contact with the technical progress of the Industry Teams necessitated regular communications. In addition to calls, email, and normal deliverables (which were few), numerous informal meetings were held on specific topics with individual Teams. While few in number during Phase I (perhaps 1 per month on each topic), these mushroomed into perhaps 1 per week on each topic during Phase II. It was hoped that use of an internet-based engineering database system would enable much of this activity to take place without travel, but this did not turn out to be the case. A combination of travel and IPDE monitoring provided adequate insight.

2.7.2 Lessons Learned

2.7.2.1 Integrated Product Design Environment (IPDE) requires a cooperative collaborative effort between Government and Industry. The IPDE works best if all the users (including the Program Office) can make recommendations on security, software selection, modem speeds, and architecture/equipment needs. This should not be construed as an "automatic" at the initiation of a design effort.

2.7.2.2 "Open orders" need to be issued to facilitate rapid response on travel. Meeting dates were in constant flux due to natural program dynamics of each Industry Team. ASJPO was constantly making last minute travel plans and changes to accommodate this. Near the end of Phase II, DARPA instituted new travel rules that required more lead time than was available for many of these meetings. To be responsive, open orders, with limitations, would greatly ease the administrative burden on the travel system. In addition, a portion of travel funding should be left at the parent organizations to expedite the travel process for those outside the Program Office.

2.7.2.3 Extensive ASJPO use of email for internal communications worked well. Portable computers allowed online access while on travel and during off-hours. A secure ASJPO Web Page and encrypted e-mail capability for source-sensitive materials would have improved the process. Secure video teleconferencing capability was underutilized but recommended for future efforts.