Quick Tweak # 1

The following screenshot shows an open requirement document with a number of requirements. Requirement 2.1.1.1  has a  lock icon with the following text Record is locked through workflow displayed under it. This indicates the requirement is in a workflow state that locks it, which prevents other users from editing the requirement.

Requirement Document with Locked Requirement

To change when requirements are locked, log in to TestTrack. Choose Tools > Administration > Workflow. Select “Requirements” from the Type drop-down list. Click the “States” tab if it is not selected.

To configure the workflow so a requirement is unlocked in a specific state, select that state and click “Edit”.  Select “Unlocked” from the Locked drop-down list and save your changes. Users will now be able to edit requirements in that workflow state.

Requirement Workflow State locked option

Quick Tweak # 2

If you use TestTrack TCM, you can also generate test cases from approved requirements. To enable this, select the “Can generate test case” option when you’re editing a workflow state.

Quick Tweak # 3

The sample workflow has a number of default states and events that can also be changed to configure the workflow to fit your needs. The following screenshot shows the sample project’s requirement workflow.

Requirement Workflow Diagram

You can modify the workflow by removing or adding additional states and events. An event moves a requirement from one state to another. When you create a new state in the workflow, you will most likely need an event to move the requirement in and out of the new state. Keep in mind that an event can also be executed and have no effect on the state of an item. As an example, the Assign event is only used to assign an item to one or more users without changing the state of an item.

In the following example, I want to change the workflow so a requirement can go directly to the Approved state from the Awaiting Review state. I’m going to use the Approve event to move the requirement.

To implement this change, log in to TestTrack. Choose Tools > Administration > Workflow. Select “Requirements” from the Type drop-down list. Click the “Transitions” tab, then click “Edit” in the State Transitions area.

Requirements State Transitions

Select the “Awaiting Review” check box for the “Approve” event, as highlighted in the following screenshot.

Requirement State Transitions Changed

After you save your changes, you’ll notice you can now move requirements directly from the “Awaiting Review” state to the “Approved” state.

Updated Requirements Workflow Diagram

Check out the TestTrack RM Resource Center to learn more about configuring TestTrack RM.

Peter Varhol, Seapine solutions evangelist, will present “Requirements and Agile: Keeping up with Change,” a presentation about how to bridge the gaps between traditional requirements and user stories. Peter will show how to reduce documentation needs while still maintaining a minimal level of more formal requirements for validation and traceability.

Alan Bustamante, senior Agile consultant for Seapine’s Agile Services group, will also be on hand to present “Only the Agile Survive: Understanding the Business Case for Agile.” Alan will discuss the bare essentials of Agile and the reasons behind the market’s move toward Agile solutions. He will also share some personal observations about successful implementations, including his experience with diverting almost $200,000 USD in project funds to another higher priority initiative, which was only possible with Agile methods.

Seapine Software will also be exhibiting at the conference in booth 405.

TechNet Aero (formerly AeroTech) is conducted annually by the Dayton-Wright Chapter of the Armed Forces Communications and Electronics Association (AFCEA) since 1990. Over 2,600 industry, government, and academic professionals interested and working in all dimensions of business and defense technologies attend. The trade show features industry and government displays and demonstrations of emerging technologies.

Q&A

Where can I find the FDA draft document for medical applications?

All of the recent medical device guidance documents can be found on the FDA site: www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm162707.htm. (Avellanet)

Is an application that sets up datasets for infusion pumps considered a medical application?

Take a look at the guidance, specifically at the criteria listed on pages 13-15, and then the examples provided by the agency in Appendix A of the guidance. (Avellanet)

Is there anything relevant to web-based medical applications for desktop computers?

See answer above, and look to other guidance documents on medical software applications. I suggest comparing what the web-based application does versus what issues are raised by FDA in the guidance documents that cause an application to “cross the line” into medical device technology. If the web-based application meets the criteria laid out by the agency, be very cautious about assuming the guidance doesn’t apply to the web-based application. After all, many mobile applications use the web to store data. (Avellanet)

How should we handle traceability if part of our product is developed by external parties?

The agency will expect you to treat the external party (assuming the external party has a different quality system than yours) as a supplier. You will then want to tailor your traceability activities and records accordingly. (Avellanet)

How do you recommend combining FDA traceability with Agile software development?

This is a great question that unfortunately can get quite complicated as it incorporates a lot of permutations. Look specifically at chapters 5 and 6 of the book. If you look at the lifecycle diagram on slides 18 and 34, you’ll see an idea of how this will work as those diagrams are from the book. (Avellanet)

It can be complicated as John mentions, but there are tools, like TestTrack, that can help support faster development methodologies, including Agile. It really comes down to vernacular (naming conventions) and data structure of users stories, ensuring you can still meet compliance objectives for verification and validation activities. Seapine’s approach allows us to be fairly flexible with development methodologies by focusing on the task while ensuring compliance activities and testing are also included through enforceable workflow and automation. The compliance piece can be transparent to daily tasks, because our tools track and document everything along the way. It’s more like a test-driven process, to verify each testable user story can be validated. The FDA does not state that you have to adopt a specific methodology, just verify and validate with objective evidence. You do not have to write all your requirements or user stories at once. You can start coding or creating specific hardware prototypes, which does allow for faster VOC feedback in clinical phases. The obvious caveat is the type of device or software you are testing or verifying with your customers/testers. John makes a great point about this in his book, which basically says design with quality in mind. If you decide to adopt Agile or elements of it, you’ll want to ensure you are approaching it with the end in mind for both for testing and what the FDA expects. It all comes down to quality. (Nicholson)

How does traceability affect my suppliers?

It is entirely dependent on what your suppliers do for you and the contractual arrangements you have with them. Two simple examples: If it’s just a packaging supplier who provides the cardboard shipping carton for the device (but not the actual box with the labels printed on it, etc.), then traceability doesn’t impact them whatsoever as the onus is on you to ensure the carton can protect your device during shipment. If the supplier is a contract manufacturer who is producing your prototypes, you need to know if they are making changes to the production process or the device. If they make changes, then someone needs to be tracking it – either you or the supplier – and that should be defined and documented in your contractual agreement. (Avellanet)

I agree with John, but in the context of suppliers or contractors who may help with development or testing, you’ll want a system that really limit them and track everything they do. We give our clients the flexibility and control to explicitly control what suppliers can actually see and do with the system, as they may not need full feature capabilities, read-only rights, hide fields, notifications, etc. For example, you may have a supplier that should be notified if a specific change happens, which you could control with a tool like TestTrack. Its security group capabilities give our clients the ability to document the permissions and explicitly control them based on what they decide. Most companies want this robust capability, because there are concerns of intellectual capital loss in addition to managing change. (Nicholson)

Can you provide any forms or templates for each of the eight basic design control steps?

All the various steps come with forms and templates refined and tailored over the years to specific clients. A generic form of the product concept sheet is available to readers of Get to Market Now! Turn FDA Compliance into a Competitive Edge. (Avellanet)

What advice would you give to an engineering undergrad that has several ideas for medical devices?

Focus on whether or not you want to be an entrepreneur. If you want to be a medical device entrepreneur, then try to find a mentor who has done this before and can help you avoid the silly mistakes that we’ve all made (you can see a tragic set of examples in this case study from my own personal history on my web site: Want a Good Outsourcing Partnership? Know How to Ruin One First). If you don’t want to be an entrepreneur, think about what you will do with your device design – sell it, partner with someone, or? Then take that answer and talk to one of your professors for advice on who he/she would recommend you talk to that has gone that route before. (Avellanet)

You said to start traceability at project go/no go. Why not at sign off of the design input?

Well, technically, the design control regulations start “Once it is decided that a design will be developed” (i.e., you’ve made a conscious decision based on concept and feasibility studies). So, in 21 CFR 820.30(b), planning for device design and development (including traceability) fall under design control. Design input approval, 820.30(c), comes after the planning, so by the time you’re at the sign-off stage, you are already under design control and should have started your traceability documentation. This is why I strongly suggest using a more formal “go/no-go” decision as you wind down the feasibility stage (i.e., preclinical), and thus I suggest using that product concept document. I also wrote a very well-received article on this–admittedly for biotechnology firms–entitled Speeding Time to Market with a Preclinical Stage Gate. Take a look at how and why I suggest structuring traceability to start at the “go/no-go” decision point. You may also want to look at the chapter in Get to Market Now! Turn FDA Compliance into a Competitive Edge that discusses improving innovation, the voice of the customer, and the product concept sheet. (Avellanet)

I agree with John, start it as soon as you start design. It’s easier than you think, if you have the right tools for design traceability. (Nicholson)

I get that the cost of non-compliance is huge, but what should the cost of compliance be as a percentage of revenues?

I have done several speeches on this exact topic. Two of the most recent public ones were Are FDA’s Quality System Requirements Bankrupting Your Company and FDA Compliance on a Budget: Avoiding FDA Trouble While Surviving Budget Cuts. You can get recorded copies on the FOI Services web site, or simply contact me to see if this would be something that might be of interest to your organization as a private webinar or onsite workshop. (Avellanet)

How deep into testing should the trace matrix cover?

As I noted in the webinar, this is going to depend entirely on the risks associated with your device, your technology, and your production (including even distribution and suppliers). I would expect a lot of detail and depth for a complex device involving both hardware and software. On the other hand, I would expect a low level of detail and depth for a simple device such as in the Class I category. (Avellanet)

I agree with John, but I have seen a trend with medical device companies moving up the classification scale. If you know you’re going to be a Class II device, based on future capabilities, three or four years down the road, start incorporating good traceability early. Instituting good traceability and testing early doesn’t have to be painful, plus it gives you better metrics and visibility. If you’re testing, but worried about documenting the testing efforts, then you likely need a tool that documents behind the scenes. (Nicholson)

I am a Quality Engineer for a major medical manufacturer, and traceability resides with R&D. In your experience, who owns traceability and which group or department is typically most effective at maintaining traceability?

Frankly, it’s all over the map – there is no one right breakdown. I will tell you what I do with my clients to help them think through this: Which group is going to be accountable to the FDA inspector for traceability versus which group is going to be accountable to senior management (given that they are the liable ones under both regulations and product liability statutes) versus which group is going to audit it to make sure the other groups are doing what they should? Clearly the agency is going to be skeptical if the same folks responsible for implementing and maintaining traceability are the same folks accountable for auditing their own work. Another really good approach is to ask your legal department who they’d like to see in charge of traceability. I’ve seen situations where everyone agreed that because engineering was closest, they were the best choice to implement and maintain it up until legal weighed in and noted that, when it came to defending the company in court or to regulators, the last thing they wanted to do was put engineering up on the stand–and thus it ended up in the hands of the quality department. It really is dependent on the dynamics of your organization and how you’ve set up not only traceability, but also the auditing of the traceability function and records. (Avellanet)

Great question and John is right, but I also think that everybody in the organization needs to take ownership, especially if they touch an item being traced. If you have integrated system, it’s not that painful. Our paper on Six Exercises to Strengthen Traceability has been popular with clients who want to start a discussion in their company about traceability ownership and best practices. (Nicholson)

What trace matrix formats does the FDA typically like to see?

There is no defined answer as long as you can provide accurate and complete records. That’s why I noted that almost all of us (myself included) started out simply with a spreadsheet or even a notebook, but it can quickly get out of control without an automated system. (Avellanet)

Agree, but an integrated automated system lets you have the option of different formats. Our tools include an Impact Analysis report, which is much easier to read than a traditional trace matrix in a table format. (Nicholson)

How detailed does a risk analyses have to be?

See the March 2011 draft GHTF guidance document Essential Principles of Safety and Performance of Medical Devices, and the July 2005 guidance Implementation of Risk Management Principles and Activities Within a Quality Management System. (Avellanet)

Where do clinical trials and all the associated documentation fit into the overall device design control scheme?

They are part of the overall design of the device. This documentation can be referenced in your traceability matrix depending on how you planned out how to test the device design. Note that I am not suggesting you input your clinical documentation into your traceability matrix, only consider linking to it or referencing it. (Avellanet)

How many times should FDA inspectors have to ask about traceability during inspection before they expect an answer?

Once. (Avellanet)

Agree. The inspection process is really about instilling confidence with the inspector that you have good controls. If you can’t answer their questions promptly, that’s a red flag. (Nicholson)

The reality is that I do not have perfect traceability at this point in my R&D. Do I go back and improve the past or start today and prepare excuses for the auditor?

Here’s what I’ve done when I’ve arrived on the scene of a company in your shoes, whether I went to work directly for them or I was called in as an outside expert either as a consultant or as an independent observer in a consent decree type of situation. Step one is to start today with the “good” documentation. In other words, start to do the right thing going forward. Then, as close as possible to starting that step one, undertake step two, which is a gap analysis of what’s missing (where specifically did we go wrong and how). Step three is then to analyze that “how” against what you started in step one (am I doing all the right things, do I need to tweak something, etc.). And then step four is, essentially, retrospective traceability (i.e., going back and trying to reconstruct as much as possible). Here’s why this approach works – it provides crystal clear documentation that says to any outside observer (say an inspector or a product liability court), “Look, as soon as I found out that I needed to do better, here are the steps that I not only took to do better, but here’s where I went back to try to fill in all these gaps that I uncovered with my new knowledge.” (Avellanet)

Start today and build an automated process going forward. If you adopt a tool that minimizes the duplication and manual effort typically required with disparate systems, you’ll be able to quickly identify gap analysis and establish traceability with links to imported data. (Nicholson)

By putting issues into the traceability matrix, is there a risk that you’re more prominently highlighting defects or bugs with the system to auditors?

If we assume that you are going to fix the issues or come up with workarounds (including design changes to avoid an issue in the first place), then all it does is provide the documentation that you took seriously your responsibility to design a safe, effective, quality medical product and the regulator expectation of “continuous improvement.” (Avellanet)

No, I’ve never seen any development cycle that didn’t have issues. Auditors are aware of issues or tasks as a result of a failed test case. I would think that a failed test case without an issue would cause more red flags with an auditor. You would obviously want to close (verify) the loop on any issue that is in an open state, so good analysis and verification needs to be conducted. An integrated system will ensure these mistakes are minimized with good automation rules. (Nicholson)

You’ve mentioned Agile might be a nightmare for traceability analysis. Can you please elaborate?

I mentioned that in the context of if you are trying to maintain traceability manually and depending on how agile you want to be. I’ve seen firms actually entertain the notion of hiring someone full-time whose sole job is to update a traceability matrix! While I think that’s great from an “employ people in your community” standpoint, I’m not sure it’s the best business decision particularly if that product might be canceled and the new hire is just as quickly out of a job. Thus, if you are going to take the agile, rapid-prototyping route (which, personally, I love), then you want to think hard about automating as much of the traceability analysis as possible. (Avellanet)

I believe it could be a nightmare, if you don’t have a flexible system that tracks all these changes and still manages all the linked relationships when moving items around. Without naming names, too many of the Agile-type tools are not doing a great job at tracking and maintaining these relationships, not to mention good validation reports. I believe our configurable and flexible tools make Agile more realistic for high-risk projects. (Nicholson)

So a project requires traceability after the concept has been approved and a budget is set? Or after a first prototype has been made? I guess our hardest decision is when is a project subject to traceability…when is it a “go” in other words?

I think in 16 years, I’ve never seen this be an easy decision. As I noted above in my answer to a similar question, the FDA expects you to start design control–and thus traceability–as soon as you are done with feasibility studies. Unfortunately, I can’t recall anyone I’ve ever worked with stating, “Okay, as of this coming Friday, we completely done with feasibility, so that means on Monday, people, we are all under design control.” It doesn’t work that way in the real world. The goal is to have about 80% of the feasibility information you need in order to make a decision. I find that the product concept document (especially when combined with a preclinical stage gate review) gets everyone focused: “Look, we have enough of the feasibility data now to decide are we going to develop this as a medical device with an intent to commercialize it – or not? Yes or no.” If it’s “Yes,” you’ll need to answer questions such as what are the resources, the timelines, who’s going to be involved in planning the development (such as clinical trials and nonclinical studies), who is going to be the regulatory affairs point person, (i.e., we are now under design control and 21 CFR 820.30)? If the answer is “No”, then you have another decision to make: Are we going to cut and run on this effort, or does it need to go back into the feasibility hopper until we are ready to say “Yes?” I don’t mean to sound so cut-and-dry, but there comes a point where one either fishes or cuts bait. As one of my old colleagues used to say, “It’s hard to be a little bit pregnant.” (Avellanet)

Agree with John, but why wouldn’t you start traceability early? There are several benefits besides compliance, like visibility, better communication, and historical data. (Nicholson)

In these instances, you may have a need to manage both requirements and user stories in the same project. In this post, I’ll talk about one way to do this in TestTrack.

First, let’s look at a sample scenario that includes managing both requirements and user stories.

Requirement Break Down

In the above diagram, you can see it starts with a Customer Requirement. The customer tells us what they want to see in our product. We then break this down into one or more Product Requirements that describe what we are going to do to address the Customer Requirement. Then, each Product Requirement is broken down into User Stories. These describe how we are going to implement the Product Requirement. Finally, each User Story is broken down into actionable tasks.

We can implement the above hierarchy in TestTrack by defining Requirement types and using the Requirement Document.

Using Requirement Types, you define the artifacts you are going to use. In the following screenshot, you can see the requirement types I have defined.

Requirement Types

Using the Requirement Document, you can establish the hierarchy and establish the necessary relationships.

Requirement Breakdown

Story points are assigned to user stories and estimated hours are assigned to tasks. For requirements, a higher level estimate is used. A popular mechanism for high level estimating is the use of rocks. For example, a large customer requirement may be assigned a boulder. A small feature could be assigned a pebble.

Once user stories are created, they can be added to the backlog of the current project. Based on the size of the product requirement, its user stories may be in multiple sprints.

Since TestTrack allows you to generate test cases from any requirement type, you can ensure that you are verifying and validating each level of the hierarchy. The links that are established between the requirements, user stories, and test cases allow for end-to-end traceability. I covered this in more detail in the Five Ways to Boost Enterprise Agility webinar a few months ago. If you would like more details on what I have covered above, I strongly recommend watching the recording of the webinar.

Peter and Mike discuss several aspects of requirements management, and why traceability is so important to the software development process these days. Following are a few interesting nuggets, if you don’t have time to listen to the 46-minute interview:

01:30 - Definition of requirements traceability

09:50 - How to make traceability automatic

21:55 - How to know if your traceability strategy measures up

30:00 - Information about managing requirements traceability with TestTrack

Have an opinion on traceability, or a story about how your team uses traceability to improve quality? We’d love to hear it—leave us a comment!

It’s really the best of times and the worst of times for the gaming industry. The top 10 titles are bigger than ever and growing, the part of the industry that is struggling is the “middle class of title.” The middle-budget and middle-rating games aren’t seeing as much interest as they did just a few years ago … There’s no way to market yourself out of a mediocre game.

His point was that quality is the key imperative to a title’s success, and that’s something we’re seeing in every industry right now. The products that best meet users’ needs—and do so reliably—are the products that win in this market.

A key challenge we’re solving for customers in this regard is improving their ability to reliably translate market needs into delivered software. On the one side, you have product management and analysts responsible for “listening to the market” and formulating the product vision. On the other side, you have development and quality assurance responsible for delivering on that product vision.

What we’re seeing in the market is a fundamental disconnect between those groups:

• Development can’t connect code with requirements, so they provide “guesstimates” to QA and management on true feature-completeness.
• Quality Assurance can’t connect testing with requirements, giving them limited visibility into test coverage and overall product quality.
• Analysts have no insight into the impact of requirement changes, which prevents making intelligent risk/reward trade-offs during the development cycle if business needs change.

The fix here is to enable traceability of artifacts throughout the development lifecycle. To learn how improved traceability can close the gap between product vision and what is ultimately delivered, register for our ‘When Requirements Change’ webinar. Peter Varhol, Seapine Software solutions evangelist, will be discussing what traceability is, why traceability matters, and how to effectively implement traceability without hurting your team’s productivity.

This long delay between understanding and actually meeting user needs is a key driver in organizations adopting Agile practices. Successful businesses and IT departments are looking for ways to better understand, and respond more rapidly to, their customer’s needs. Whether they call it “Agile” or just “good business sense,” the goal is the same—get closer to customers to more quickly deliver products that better meet their needs.

Unfortunately, users and developers haven’t gotten much better at translating needs into working software. Techniques like UML have improved that process, but fundamentally the user needs to “use” the software to gauge it’s effectiveness. As users, we don’t always know exactly what we want. That’s why we still test drive cars, try on new clothes at the store, and evaluate software for 30 days. And therein lies the challenge; how to support frequent change from your customer without blowing up the schedule, abandoning quality goals, or driving the team mad with rework?

The short answer is traceability. Traceability is about connecting related development and testing artifacts, enabling targeted and timely communication across the team as change happens. When the customer tweaks the user interface requirements at the next usability lab, those changes can be immediately communicated to the developers and testers responsible for the UI. When the customer realizes that their entire team, not just a few super-users, will need to access the web application you’re building, updated performance and load requirements can be communicated to the development and testing teams.

My colleague Peter Varhol, a Solutions Evangelist here at Seapine, will be talking more about traceability and how to effectively handle requirements change on Thursday, February, 24. If you’re struggling to increase the pace of software delivery without sacrificing quality or schedule, take a minute to register for this webinar to learn more about traceability and how it can enable closer collaboration with your customers!

I usually ask a few questions before responding:

• Are there multiple document approvers?
• Does each document approver have an area of expertise?
• Does the approver verify the content of each and every requirement in the document?
• Is there a need to have accountability in specific areas of the requirements process, such as design, technical, or software?

What I find is that customers usually ask about approving all the requirements in a requirement document at once because they’re used to working with requirements as a single Microsoft Word document. Requirements have often been written as paragraphs that include multiple functions of a product. Because these requirements are in paragraph form and not broken down into manageable and actionable data units, customers find themselves routing entire requirement documents to large groups of people in the organization for review. The reviewers are subject matter experts (SME) who have backgrounds in a multitude of disciplines, but their true expertise typically only covers a small subset of the requirements in a document.

So after asking my questions, understanding the history behind the customer’s request, and discussing it with the customer I usually hear back “it would be great if each group only received the requirements that they needed to approve.”

And indeed, with an integrated requirements management solution you can do just that. The first step, and often the most challenging when you’re just getting started, is to break the requirements document down into more manageable requirements. Your goal is to get to a point where each requirement covers an individual function point in the product. Breaking the document into smaller requirements allows you to assign an “owner” to each requirement, effectively defining who is responsible for each requirement in the document.

When you start managing at the individual requirement level you also improve traceability within the project. You can now process a single requirement through your workflow, evaluate the project-wide impact of changing that requirement, and generate one or more test cases tied to that one requirement. In the end, you’ve improved your ability to verify that the delivered product meets the requirements from which it was built.

In summary, managing individual requirements instead of one big document allows you to send requirements to the appropriate SME for sign off, reduce the volume of requirements that each SME reviews, and increase the quality, accuracy, and efficiency of meeting project requirements.

This one-day event, near Redstone Arsenal, will focus on software solutions that help resolve the change management and collaboration issues facing IT and software development teams. You’ll hear how Seapine customers have handled some of the same issues you may be facing. Seapine experts will also be on hand to discuss development, testing, and project management.

• Improve quality, monitor compliance, reduce risk, and deliver projects on time
• Manage traceability and compliance across requirements, development, and testing with Seapine’s integrated suite of ALM tools
• Use an integrated lightweight approach to project management to become more agile throughout the application lifecycle
• Measure development lifecycle efficiency with enterprise reporting

You’ll also learn what’s new in upcoming releases of Seapine’s TestTrack and Surround SCM.

This is a free event, but space is limited. Register Now!

P.S. We’re planning more stops for early 2011, leave a comment if you’d like to see us come to your city!