U.S. Flag Official website of the Department of Homeland Security

Note: This page is part of the us-cert.gov archive.This document is part of the US-CERT website archive. These documents are no longer updated and may contain outdated information. Links may also no longer function. Please contact info@us-cert.gov if you have any questions about the US-CERT website archive.

TLP:WHITE

Training and Awareness

Published: September 26, 2005 | Last revised: August 07, 2013

Author(s): Carol Sledge and Ken van Wyk Maturity Levels and Audience Indicators: L3  / M  SDLC Life Cycles: Deployment Copyright: Copyright © Carnegie Mellon University 2005-2012.

Abstract

This article provides guidance on training and awareness opportunities in the field of software security. It examines the state of the practice of commercial/non-profit software security training and awareness offerings and makes recommendations for goals and curricula contents.

Overview
State of the Practice—Commercial
State of the Practice—Academic
Best Practices in Training and Awareness
Measuring Knowledge
Business Case
Glossary
Commercial Training Examples
Academic Curricula Sampling

Overview

This document is intended to provide guidance on training and awareness opportunities in the field of software security. It examines the state of the practice of commercial/non-profit software security training and awareness offerings and makes recommendations for goals and curricula contents.

An effective training program is vital to adopting new software development practices. And, because software security is still an emerging and rapidly changing field, there are few experienced developers who are familiar with the sorts of practices described here in the Build Security In (BSI) portal. As such, a clearly defined training and awareness campaign is particularly important for this effort. Indeed, Microsoft considers developer training to be a vital component of its Security Development Lifecycle (SDL), and makes annual training a requirement for all of its software engineers [Howard 2006].

A commonly heard gripe in the industry is that academic curricula do not adequately address software security issues. We take a brief look here at offerings from universities and colleges in the United States, in particular those associated with the National Security Agency (NSA) and Department of Homeland Security (DHS) Centers for Academic Excellence in Information Assurance. When possible, it is a good idea to guide training and choice of academic programs by a recognized common body of knowledge (CBK). The DHS-sponsored Software Assurance Curriculum Project (SwACP) in 2010 released a Master of Software Assurance (MSwA) Reference Curriculum, which contains a CBK [Mead 2010]. The curriculum is recognized by the IEEE Computer Society (IEEE-CS) and Association for Computing Machinery (ACM) as appropriate for a master’s program in software assurance. This formal recognition signifies to the educational community that the MSwA Reference Curriculum is suitable for creating graduate programs or tracks in software assurance. February 2012 saw the release of the strawman draft of the Computer Science Curricula 2013 (CS2013) by the ACM and the IEEE-CS [Sahami 2012]. Information Assurance and Security is one of two new knowledge areas in CS2013. Information Systems Security Certification Consortium, Inc. (ISC)²® develops and maintains the (ISC)² CBK, a compendium of information security topics.

State of the Practice—Commercial/Non-Profit

To assess the state of commercial training offerings available today, we examined the publicly available documentation, syllabuses, etc. from numerous commercial/non-profit organizations. We looked for training that emphasizes as much of the BSI concepts as possible, including the best practice activities, knowledge base topics, and available tools. We also looked for training that is largely process agnostic, as are the concepts laid out in BSI.

The good news is that there are indeed many offerings to choose from. They range in size and scope, and they cover a broad spectrum of aspects of software security. The biggest strength in the available courses is that most of them provide a good amount of detail on the technical nature of current problems and available solution sets. There are also more role-based offerings, such as the Certified Secure Software Lifecycle Professional (CSSLP®) offered by (ISC)².

That said, the bad news is that a number of the available commercial courses appear to suffer from various shortcomings, at least with regard to the approaches presented in BSI. What follows is a brief description of those shortcomings, along with recommendations on how to avoid or alleviate them.

  • Security software vs. software security. According to their syllabuses, many of the software security training offerings spend a great deal of time describing security functionality (the use of encryption, identification and authentication mechanisms, etc.). Although these security functions are vital ingredients of software security, they are essentially security ingredients. The topic of software security goes far beyond a simple list of security ingredients.
  • Knowledge vs. practices. Similarly, many of the training offerings focus on individual problems (e.g., buffer overflows) and their respective point solutions. Avoiding these pitfalls is also vital to writing secure software, but much else needs to be covered for the training to be as effective as it needs to be.
  • Network and operating system focus. Although not as common as the above two shortcomings, some of the available training offerings appear to present a network and operating system centric point of view to the issues of developing secure software.

While all of the above elements are important to cover in a software security training program, what we feel is principally lacking in a number of the course offerings is adequate coverage of security processes that are necessary to incorporate software security into the development processes and practices. These should be at least similar to those presented in the Best Practices area of BSI. Further, an emphasis should be placed on some high-value best practice activities such as abuse case analysis, architectural risk analysis, risk-based testing, and code review practices and tools.

At least the situation is steadily improving, with numerous high-quality courses recently becoming available. There are improved courses and certifications, and, as mentioned earlier, the more holistic software development security assurance process embodied in Microsoft’s Security Development Lifecycle. SDL embeds security processes into the software development lifecycle to make it more secure. These security processes include, “security requirements, threat modeling, static analysis, dynamic analysis, security review and product incident response” [Chess 2012]. Additionally, Microsoft makes SDL information and tools available to (external) developers on its website and blog.

The Denim Group has donated its ThreadStrong secure software development courses to U.S. universities to help students learn how to build more secure software.More information including how universities can apply for a complimentary license is available on the CERT website.

The software assurance community as a whole is building upon earlier efforts and provides information, methods and frameworks, which can be used by individuals and organizations, and incorporated into course offerings (non-academic and academic.) According to “The Software Industry’s ‘Clean Water Act’ Alternative,” Robert A. Martin and Steven M. Christey the Common Weakness Enumeration (CWE)CWE is co-sponsored by MITRE and the National Cyber Security Division of the U.S. Department of Homeland Security. offers the industry a list of potentially dangerous software contaminants, and the Common Weakness Scoring System (CWSS) and the Common Weakness Risk Analysis Framework (CWRAF) provide a standard method to identify which of these contaminants are most harmful to a particular organization, given the software’s intended use. CWE, CWSS, and CWRAF efforts are being adopted by the community and integrated into static-analysis offerings and other solutions” [Martin 2012]. Among others, “SANS publically declared support and plans to incorporate CWSS. Likewise … EC-Council…and OWASP all declared their plans to work on using and evolving CWRAF to meet … community needs for a software error scoring system”[Martin 2012].  The Top 25 CWEs represent the most significant exploitable software constructs that have made software so vulnerable. Addressing these will go a long way in security software, both in development and in operationSee the list of Top 25 CWE Most Dangerous Software Errors on the Software Assurance Community Resources and Information Clearinghouse website for more.. The CWE site also contains other information for self-directed training.

State of the Practice—Academic

Within academia, a number of colleges and universities are now offering optional senior-level undergraduate and graduate courses in software security. These courses tend to be broader in focus than their commercial/non-profit counterparts, in that they include discussions on the sorts of best practice activities mentioned above in addition to discussions of common vulnerabilities. Additionally, a number of community colleges may also offer courses in this area. A number of these community college offerings are aligned with commercial offerings and/or certifications. Departments and programs such as Computer Information Systems are likely to teach applications development. To facilitate more community college offerings, the Department of Homeland Security (DHS) sponsored the Software Assurance Curriculum Project (SwACP), which has produced a report [Mead 2011] and article [Hawthorne 2012] focusing on community college courses for software assurance. The courses are intended to provide supplementary education for students with prior undergraduate technical degrees who wish to become more specialized in software assurance or to provide students with fundamental skills for continuing with graduate-level education. Since community colleges exist to serve their immediate constituencies, industry may find faculty and programs willing to incorporate additional secure software topics and courses. Community colleges are of particular importance, as many practitioners utilize these (less expensive) community college offerings for additional training—choosing courses based on their particular needs, and not necessarily with the intent of obtaining (another) degree or certification. 

With respect to four-year undergraduate and graduate schools, one excellent source of programs are those academic institutions that have obtained (and are currently designated as) Centers for Academic Excellence (CAE) in Information Assurance Education (IAE). This designation is jointly sponsored by the National Security Agency (NSA) and the Department of Homeland Security (DHS) and awarded for a period of five years. Recently CAEs in IA two-year Education and Training were added. The currentAs of July 3rd, 2012. list of CAE/IAE academic institutions (those in 42 U.S. states, plus the District of Columbia and Puerto Rico) can be obtained at the NSA website.

As an example, selecting Stevens Institute of Technology (NJ) from that list on the NSA website will take you to the Center for the Advancement of Secure Systems and Information Assurance (CASSIA). Scrolling down to “Academic Programs” will show a list of undergraduate degree programs, master’s programs, doctoral programs, and graduate certificates available. Knowing the names of the programs and certificates can help locate the specific information in the various departmental sites.

As was the case for appropriate community college courses, some professionals may choose to take particular courses for continuing professional development/education purposes. For those in senior level positions or those who already have advanced degrees, it may be that they choose to take courses for professional enhancement, rather than matriculate for a degree or certification. 

There is certainly room for optimism in these findings, while at the same time there is also room for improvement. For example, a strong argument for integrating discussions of secure design processes, avoiding buffer overflows, and similar topics into the more general computer science courses could easily be made; why not teach students to avoid strcpy() and the like in an Introduction to C course? Integrating software security into the entire curriculum is bound to be more effective than offering it as a senior-level elective course.  Fortunately, some faculty are doing this. For example, Dr. Steven Hadfield and other faculty within the Department of Computer Science at the U. S. Air Force Academy have used a cross-curricular approach to integrate software assurance and secure programming concepts across their course offerings [Hadfield 2011, 2012]. Additionally, there are other efforts to provide faculty with materials. Dr. Blair Taylor of Towson University led a group of educators to produce modules related to security injections. "Security injections are strategically-placed security-related modules for existing undergraduate classes. The combination of lab exercises and student-completed checklists in these security injections has helped us teach security across the curriculum without adding extra pressure on already-overburdened undergraduate degree programs"This work was funded under National Science Foundation grant DUE-0817267. [McKay 2012]. These materials are available at the Towson University website

For some organizations, it may make sense to partner with academic institutions or faculty to offer specific courses, training, etc. Large organizations may decide to go even further. Recently Northrop Grumman and the University of Maryland teamed to “develop a curriculum to produce graduates who can enter the workforce with an eye toward cybersecurity defense.” Northrop Grumman will provide financial support and, in the fall of 2013, the university will “develop 45 students per year in the Advanced Cybersecurity Experience for Students (ACES) program. The students, from majors including computer science, engineering, business, public policy and social sciences, will live together in a “learning-living” program and use state-of-the-art laboratories in the yearlong capstone project” [McKay 2012].

Best Practices in Training and Awareness

As stated above, we feel that a best practice software security training program today should encompass the various best practices, knowledge bases, and tools presented in BSI. Further, training and awareness initiatives should plan for—at a minimum—three target audiences: senior decision makers, engineering managers, and software developers. Each of the audiences should receive training that addresses its needs, naturally. The most fundamental goals and objectives for each audience follow.

  • Senior decision makers

    For a software security initiative to succeed in an organization, the organization’s senior decision makers need to support the initiative with their buy-in. Therefore, an awareness training program should be presented to them that clearly articulates the need for software security and the difficulties faced in delivering secure software. The training should also succinctly describe the best practices necessary to accomplish those deliveries.

  • Engineering managers

    Likewise, engineering or software development managers (across all of the organizations and disciplines involved in the overall development process) also need to buy into a software security initiative. Additionally, however, they need to have a thorough understanding of the software security practices that their organization will be incorporating into its development processes and specifically what their sub-organizations will need to do as a result. This is essential for managers to understand for purposes of project planning and execution. Thus, managers should be provided training content that describes the need for software security, as well as a thorough description and understanding of their organization’s software security practices. The training should emphasize, where possible and feasible, the levels of effort for each software security activity involved, how to identify areas for improving existing software security practices (e.g., evaluate and improve), as well as methods for measuring a development organization’s software security effectiveness. It may be beneficial to tailor the training content by audience somewhat—e.g., development managers are likely to have different specific needs and interests than test and quality assurance managers. In this case, the disciplines that each audience are directly responsible for should be emphasized and covered in more detail than the others.

  • Software developers

    Software developers should receive training that provides them with a conceptual foundation of software security, its importance to their organization, and the practices used within their organization. They should gain practical knowledge about the benefits of software security. Additionally, developers should receive technology-specific security training in each of the technologies that are involved in designing, coding, and testing software in the technologies that they work with. In the same manner that the training for the management may be tailored by the audience’s disciplines, the content for software developers must emphasize the aspects of software development that they work on directly. Further, it should be replete with code examples and hands-on exercises/labs to help reinforce the course material [Howard 2006].

With these goals and objectives in mind, the following outlines are presented as guidelines for developing organizational curricula for software security training.

Training Courses and Outlines by Audience

For each of the three audiences, it is particularly useful to clearly address the rationale for each security activity. Where feasible, consider demonstrating the activity through exercises, examples, and in-depth anecdotes from case studies.

Software security awareness training for senior decision makers should look similar to the following:

  1. Introduction to software security problems

    This training module should present an overview of the security problems faced today by software developers. Its aim should be to convince the audience why traditional and largely separate approaches to information security and software development are flawed from a software security perspective. Further, it should present an accurate business case that weighs the often conflicting goals of development and security.

    1. Shortcomings of traditional perimeter-based network security solutions

    2. Common software weaknesses

    3. Balancing the different goals of security and software development

  2. Software security activities to integrate into the SDLC

    This module should provide a basic conceptual overview of software security activities and their impact on the SDLC for the senior decision maker audience. It should principally focus on describing the activities and their associated costs—monetary and schedule.

    1. Requirements and specifications activities

    2. Design time activities

    3. Implementation activities

    4. Test planning and testing

    5. Deployment, operations, and maintenance issues

Software security awareness training for engineering management, as stated, should be substantially similar to that provided to the senior decision makers, but with a somewhat different core message. Specifically, instead of solely aiming to convince the audience of the merits of software security, it should also ensure that the managers have the necessary knowledge to implement and measure an appropriate set of software security practices. Their training outline should look similar to the following:

  1. Introduction to software security problems

    This training module should delve into the security problems faced today by software developers. Its aim should be to convince this management audience why traditional and largely separate approaches to information security and software development are flawed. Further, it should present an accurate business case that weighs the often conflicting goals of development and security. For the engineering management audience, particular attention should be paid to cost vs. benefit information, as they’re often the people within an organization that are the most skeptical about the benefits of adding more activities to the SDLC process.

    1. Shortcomings of traditional perimeter-based network security solutions

    2. Common software weaknesses

    3. Balancing the different goals of security and software development

  2. Software security activities to integrate into the SDLC

    This module should be the core of the training content provided to the engineering management audience. For the managers, particular focus should be given to describing the processes involved and how to implement them, as well as methods of measuring their teams’ progress and successes. Additionally, realistic program management information should be provided, such as scheduling issues and typical level of effort required for each activity.

    1. Requirements and specifications activities

    2. Design time activities

    3. Implementation activities

    4. Test planning and testing

    5. Deployment, operations, and maintenance issues

Many of the same topics covered in the above two training curricula should also be covered for the software developer audience; however, the focus here should shift dramatically from the conceptual to the technical. A conceptual foundation must be presented, but the developers will need specific technical information in order for them to do their expected software security tasks. Additionally, where feasible and possible, hands-on exercises should be incorporated into the training so that the developers can experiment with putting into practice the processes described in the training material.

  1. Introduction to software security problems

    This training module should delve into the security problems faced today by software developers. Its aim, quite simply, should be to convince the students that they should care about software security in their work. (Realistic case studies can be highly beneficial here.)

    1. Shortcomings of traditional perimeter-based network security solutions

    2. Common software weaknesses

    3. Balancing the different goals of security and software development

  2. Software security activities to integrate into the SDLC

    This module should present the same basic concepts that were presented to the engineering managers, but the principal focus should be on actionable recommendations for the developers. The students should come away with a clear understanding of where they fit into the software security program within their organization, what is expected of them, and how they need to implement software security. Specific guidance and recommendations should be provided for each of these processes that will help the student ”internalize” the correct behaviors as they apply to software security activities.

    1. Requirements and specifications activities

    2. Design time activities

    3. Implementation activities

    4. Test planning and testing

    5. Deployment, operations, and maintenance issues

  3. Know the enemy

    To build software that can withstand attacks, it is essential to understand the nature of the anticipated attacks and the concepts behind them, and in considerable technical detail. This module should teach developers about their adversaries. The students should understand who wants to attack their software, why, and how they are likely to go about doing it. Common concepts such as buffer overflows, SQL injection, cross-site scripting, and so on should be thoroughly described and, where feasible, included in hands-on exercises/labs so that the students can best internalize the course material.

    1. Threat analysis – who are the attackers and what motivates them

    2. Common software vulnerabilities explained in detail – architectural flaws as well as implementation bugs

    3. Attack tools and methodologies

  4. Knowledge base and tools

    Whereas the previous module stresses the best practices that developers are to follow, this module should arm the students with the necessary knowledge base and understanding of the tools necessary to their jobs. The actual content delivered here to each student may need to be further refined to the exact discipline of the student audience. For example, software designers need to focus on the architectural risk analysis processes and specific methodologies, whereas coders need to focus on code review methods and tools.

    1. Risk analysis techniques (e.g., STRIDE, SQM, CLASP)

    2. Language-specific tips, pitfalls to avoid, rules, and guidelines

    3. Tools for code analysis, testing, etc.

  5. Code remediation

    With the fundamentals out of the way, it is important to include training that includes prescriptive information on how to design and implement safe software. The content should be replete with code examples and should be specific to the languages, frameworks, etc., in use by the developers. At a minimum, the courseware should include example design and code patterns addressing the most egregious bugs and flaws found in similar architectures and languages. It should include the OWASP Top 10 for web developers, for example.

    1. Top 10 security defects (by technology)

    2. Safe coding examples and guidelines

    3. Exercises/labs to reinforce

  6. Security testing

    An additional training topic that is often overlooked is security testing. Security testing practices in far too many of today’s software development organizations consists of little more than a late-cycle penetration test. As we’ve seen in Adapting Penetration Testing for Software Development Purposes, this approach is inadequate. One step in adopting better testing practices is to train the development and testing team on how to do security testing in depth [Howard 2006].

    1. Fuzz testing

    2. Penetration testing

    3. Run-time verification

Of course, the above outlines are quite simplistic and generic views of the topics to be covered. Additional examples of course are cited in the list at the end of this article, as well as in [Howard 2006]. Additionally, some worthwhile considerations in creating or selecting appropriate course material might include the following:

  • Beyond the basics, courses should be as specific to the development environments in use as possible.
  • Courses should include hands-on exercises/labs whenever feasible.
  • Course material should cite common defects with specific code examples.
  • Secure coding guidance should be prescriptive with ample code pattern examples for the students to study from.
  • Instructors with exceptional communication skills are vital, but also look for instructors with hands-on software development experience.

Once a firm conceptual foundation has been laid for the students, a library or repository of up-to-date reference information should be made readily available to them. This should include external sources of information such as books and published papers, as well as internal sources such as (security vetted) design architectures, design documents, and source libraries.

Measuring Knowledge

A serious training initiative should take steps to verify and validate the students’ knowledge base, but many questions quickly emerge [Howard 2006]. There are a number of software security certification programs in the commercial and non-profit marketplace, which helps to address this issue. There is also a need to validate “the skills of individuals being trained so that not only do they receive a certification, but we also know their specific quantifiable skills, from the technical to the analytical” [Kwon 2012]. At a minimum, however, development organizations should mandate training attendance and record employee participation. If the organization tracks other related security metrics (e.g., code defect density per thousand lines of code), it may consider trying to correlate course attendance with defect density, but that is a degree of measurement that few organizations can achieve.

Business Case

As the practice of software security catches on and grows throughout the software development community, training and awareness initiatives are vital to adoption among developers and managers alike. This is particularly the case as few professional software developers today have undergone anything more than rudimentary on-the-job exposure to software security issues, and probably not much in the form of academic instruction.

For software security best practices to be successfully adopted in industry, there must be senior-level buy-in. This can be accomplished in a number of ways, including a clear and concise awareness training program that presents senior decision makers with the issues and tradeoffs involved in delivering secure software.

Further, mid-level engineering management needs to be aware not just of the issues associated with delivering secure software but with the software security best practices that they should be incorporating into their groups’ development processes and methodologies.

Lastly, software developers themselves, from architects and designers through coders and testers, need to be thoroughly trained in all of the above, plus all of the technology specifics involved in designing, coding, and testing software in the technologies that they work with.

Without these things, it is highly unlikely that software security initiatives can succeed in a substantial way. Trying to accomplish a software security agenda from a ”grass roots” or ”bottom up” perspective is not likely to accomplish more than superficial change.

Glossary

incident

Any real or suspected adverse event in relation to the security of computer systems or computer networks.

Non-Profit Training Sampling

International Council of E-commerce Consultants (EC-Council) http://www.eccouncil.org/

International Information Systems Security Certification Consortium, Inc., (ISC)² https://www.isc2.org/

The SANS Institute, Inc. http://www.sans.org/

References

[Chess 2012]

 

Chess, Brian & Wysopal, Chris. “Software Assurance for the Masses.” IEE Security and Privacy 10, 3 (May-June 2012): 14-15.

[Hadfield 2011]

 

Hadfield, Steve; Schweitzer, Dino; Gibson, David; Fagin, Barry; Carlisle, Martin; Boleng, Jeff; & Bibighaus, David. “Defining, Integrating, and Assessing a Purposeful Progression of Cross-Curricular Initiatives into a Computer Science Program”, Proceedings of the 41st ASEE/IEEE Frontiers in Education Conference. Rapid City, SD,  Oct. 2011. IEEE, 2011.

[Hadfield 2012]

 

Hadfield, Steve, “Integrating Software Assurance and Secure Programming Concepts and Mindsets into an Undergraduate Computer Science Program,” Proceedings of the 16th Semi-Annual Software Assurance Forum. Arlington, VA, Mar. 2012. Build Security In, 2012.

[Hawthorne 2012]

 

Hawthorne, Elizabeth K. " Infusing Software Assurance (SwA) into Introductory Computer Science Curricula." Build Security In. https://buildsecurityin.us-cert.gov/bsi/articles/knowledge/education/1400-BSI.html (2012).

[Howard 2006] 

 

Howard, Michael & Lipner, Steve. The Security Development Lifecycle: SDL: A Process for Developing Demonstrably More Secure Software. Redmond, WA: Microsoft Press, 2006 (ISBN 977-07356-2214-2).

[Kwon 2012]

 

Kwon, Mischel; Jacobs, Michael J.; Cullinane, David;, Ipsen, Christopher G.; & Foley, James.  “Educating cyber professionals: A view from academia, the private sector, and government.” IEEE Security and Privacy 10, 2 (March-April, 2012): 50-53.

[Martin 2012]

 

Martin, Robert A. & Christey, Steve M. “The Software Industry’s ‘Clean Water Act’ Alternative.” IEEE Security and Privacy 10, 3 (May-June 2012): 24-31.

[McKay 2012]

 

McKay, Jim. “Cybersecurity Curriculum on Tap for University of Maryland Students.” Emergency Management. http://www.emergencymgmt.com/training/Cybersecurity-Curriculum-University-Maryland-Students.html (2012).

[Mead 2010]

 

Mead, Nancy R.; Allen, Julia H.; Ardis, Mark; Hilburn, Thomas B.; Kornecki, Andrew J.; Linger, Rick; & McDonald, James. Software Assurance Curriculum Project Volume I: Master of Software Assurance Reference Curriculum (CMU/SEI-2010-TR-005). Software Engineering Institute, Carnegie Mellon University, 2010. http://www.sei.cmu.edu/library/abstracts/reports/10tr005.cfm

[Mead 2011]

 

Mead, Nancy R.; Hawthorne, Elizabeth K; & Ardis, Mark. Software Assurance Curriculum Project Volume IV: Community College Education (CMU/SEI-2011-TR-017). Software Engineering Institute, Carnegie Mellon University, 2011. http://www.sei.cmu.edu/library/abstracts/reports/11tr017.cfm

[Sahami 2012]

 

Sahami, Mehran & Roach, Steve. “CS2013 Strawman Curriculum Standard now available”.  Computing Education Blog. http://computinged.wordpress.com/2012/02/20/cs2013-strawman-curriculum-standard-now-available/ (2012).

 


Back to Top