1 in 4.
This is how many businesses will lose competitive ranking due to a lack of digital competence by 2017.
Mobile applications are now necessary to compete for today’s consumers. The businesses who ignore this message do significant damage to their brand perception in the eyes of those consumers, who have become much more fickle creatures in the last ten years. According to Accenture, 56 percent of consumers surveyed said that the number of brands they consider for a purchase has expanded significantly in the last decade. Coupled with the research showing that 85 percent of consumers prefer mobile apps to websites, it’s clear that abandoning consumers in the mobile space guarantees consumers will abandon your brand.
For employees, mobile apps are no less important than those made for the consumer audience. The strides made in reducing operating costs through the use of digital innovation have opened the floodgates of the return on investment of strategic information technology. Nearly every industry is capitalizing on the advancements of cloud computing and big data to see the bottom-line relief that can insulate them from digital disruptors taking over the industry.
Simply put: the world has changed for mobile apps.
The Real Requirements of Today’s Mobile Apps
The problem is that the way that many businesses pursue vendors has not changed: dust off an RFP template, fill in some requirements, and put out a call for proposals. Prioritize submissions based on cost and estimated ability to deliver, and the project is on its way.
There is too much at stake for these products, along with too much potential to transform your business to make a bad investment. The CEOs who have realized this will allocate a larger portion of spending to developing mobile solutions, but these investments have a difficult track record: 60 percent of software development produces ineffective or substandard products, with 25 percent failing outright.
Traditional RFPs are especially prone to produce products in both the ineffective and failure categories. The author of the requirements is often an IT leader with little understanding of the business or user needs in which the problem will operate. Their only mission is on-time and on-budget, without taking into account what success means beyond these two metrics.
For most companies, the root cause of this failure occurs at the very beginning. By failing to properly investigate and diagnose if a problem exists, they develop a solution based on assumptions about users’ behavior and by the time they realize the mistake, it is too costly to fix.
A NASA study revealed that at each stage of the product’s development, changing the requirements generates a ten-fold increase in cost. The further along companies go without user feedback, the more difficult it is to make the changes that users need. Many enterprises mistakenly assuming that mandating employees use the app is the solution for internal applications. Without robust user research to validate that the app works well for users, companies are only setting themselves up for failure when their users resort to workarounds, or worse, experience frustration and complication from the app.
This alone should convince IT and business leaders to abandon traditional RFP templates. With so much at stake, companies are binding themselves to an ineffective or failed solution; by the time the realization occurs that the product will not meet business KPI expectations, the budget for fixing those has run out. Every application receives user feedback; whether through a low adoption rate or the voices of frustrated users, the IT team will hear what changes need to be made. The difference with our approach to RFPs is that the team will have the budget and time to correct the user behavior assumptions that lead to those mistakes before the app goes into costly design and development phases.
Our approach is a different type of RFP: one that ensures business ROI and better suits the way truly innovative digital products are created.
The Foolproof Strategy to Gathering User Requirements for Mobile Applications
Sales had stagnated for a major foodservice product distributor. It took 50 percent more interactions for new sales hires to achieve the same results as their veteran counterparts, due to inexperience, management of sales collateral and inability to field customer questions.
The leadership team sounded the alarm for the IT team, who sent out an RFP for a customized digital sales enablement solution.
The problem: it was never a collateral issue. Observing the sales team in action yielded the insight that the customers’ moods were to blame. They were in the dark about when their previous orders would arrive and anxious about filling their own performance goals for the day.
The sales enablement solution would have never improved the sales metrics. Building a robust user research engagement with field observation into the project uncovered this in time to invest the IT dollars into a customer-facing application with real-time insights on their orders.
Traditional app development agency evaluation processes miss the user research engagement all too often. The IT leader of the project parses out the problem from the business case, does some validation of its existence and moves onto solutions.
In the medical field, the act of prescribing a solution before verifying the problem is called “malpractice.” Yet, in the IT field, this is exactly what agency evaluation processes are designed to do without a user research engagement.
User interviews are simply not enough to provide the level of detailed diagnosis that digital products require. As the process becomes more ingrained and mindless for the employee or customer, the less they are able to explain each step in sufficient detail. Process diagrams and user manuals fall victim to the same problem. Management incorrectly assumes that tasks happen exactly as they are outlined in these publications, yet reality is often very different.
Before designing a single wireframe, it’s necessary to go out to the field to observe users and then create a real workflow map to identify where inefficiencies or user frustrations exist. Use those observations to formulate a hypothesis of the impact of a digital product. Some enterprises have an existing process in place as part of creating the business case. This may still not be enough, depending on how well the user and business goals are understood.
Diagnosing if you need user experience research:
- Do you have detailed personas for each user outlining their motivations, fears and frustrations?
- Have you taken the time to investigate the environmental conditions of users?
- Have you mapped the workflows of each user to a high degree of accuracy?
If those questions can be answered and validated by data from contextual inquiry out in the field, it is best to extrapolate clear and specific user needs from those and provide a summary of the research for the RFP. If not, swap out the traditional “Summary and Scope” section for this:
- Summary of business problem. Write out the conditions that are leading to this RFP. Be as specific as possible.
- Timeline and goal launch date.
- What will the vendor do to minimize risk of product failure before the design and development phases? Discover how the team approaches risk and user experience research.
- How will your product development process validate that this product will work for users before development? The goal here is to discover how user feedback is incorporated into the product development process.
- How and when will the impact of the application be measured? Find out how the team approaches data and metrics.
- How will you ensure the product maximizes user adoption? Elucidate how the agency measures and defines user adoption. Watch out for answers that focus on the “look” of the app; plenty of beautiful, elegant apps fail to attract users.
- What are the backgrounds of the user researchers on the team? Too often, agencies rely on designers to do this, which is problematic because they lack formal training in field research.
Solid Business KPIs
On-time and under budget.
Looking at the majority of traditional RFPs, these two are the major KPIs for vendors to meet. Part of this stems from thinking of mobile solutions as projects, not products. “Project mentality” creates products without the measurable effects for users and with a mission to complete, rather than ROI.
Think of it this way: a chef at a three-star Michelin restaurant prepares a five-course prix fixe menu. While the creation is required to fit within the restaurant’s budget and be timed accordingly, neither of those will matter if it fails to satisfy customers.
Here are some useful calculations for measuring and justifying KPIs:
- Errors: (# of errors) x (avg. repair time) x (employee cost) x (# of employees) = cost savings
- Cost of Development: (# of changes) x (avg. hrs/change) x (cost of developer) x (4, if late) = cost savings
- Productivity: (time saved) x (employee cost) x (# of employees) = cost savings
- Retaining more customers:
- Retention rate= (number of customers at end of period – number of new customers acquired during period)/ customers at start of period
- (Goal retention rate x current number of customers – current retention rate x current number of customers)x cost of customer acquisition= cost savings
- New customers: (Demand generation costs after launch)/(Number of customers after launch) – (Total cost of marketing and sales activities for demand generation for previous time period)/ Number of customers= Cost savings
- Improving employee retention
- Retention rate= (number of employees at end of period – number of new employees hired during period)/ employees at start of period
- (Goal retention rate x current number of employees – current retention rate x current number of employees) x average cost of employee acquisition and training= cost savings
The key is to root both of these in the needs of the users. All of these calculations need to apply to the individual user level.
The RFP itself should include a hypothesis of the KPIs that the business would like to meet with this solution. These should be specific, measurable and timely goals that the mobile app needs to accomplish. Avoid vague or “soft” goals, such as “improve employee morale.” Instead, write out clearly defined metrics, such as “increase employee Net Promoter Score from 56 to 64 by Q2.”
For this portion, it’s important to clearly identify all stakeholders in the project and conduct interviews to outline the metrics for success from their perspective. For example, the Plant Manager has identified the need for a solution to mobilize and digitize quality control forms, which are regulated and required by a government regulation agency. The Plant Manager’s KPI is a decrease of quality control-related delivery delays and decrease in amount of time it takes a product to move through the quality control process. The coordinator responsible for managing the paperwork will have a KPI of decreasing the errors in quality control forms and decreasing the amount of time it takes to submit the paperwork.
Questions to include in the RFP:
- Description of KPIs. What does success mean to the product?
- How will you use data in the design and development processes to ensure the product is on track to meet ROI goals? Discover how hard data comes into the process.
- How will you course-correct if the project is not on track to meet KPIs? The key here is that the vendor pursues an investigation as to why the project will not meet KPIs, rather than just blindly making assumptions about the root cause.
The Non-Functional Requirements That Are Critical to the Success of Mobile Apps
Design needs to be more than ‘pretty.” In fact, the aesthetic feelings you and your superiors have about a company’s portfolio should not factor into your evaluation of vendors, unless to eliminate the vendors whose applications are clearly unprofessional or have mistakes.
First, it’s important to expand your own definition of user experience beyond just their experience with the digital product. User experience focuses on people, thinks beyond digital solutions, and is how technology creates a holistic experience that ties all the points that a user comes in contact with together to improve employee performance. UX is not wireframes, an interface, usability experience, or technology alone. The term has evolved and now considers the holistic experience around a software product, as well as the brand and environment. In addition, user experience design and experience design are melding together since artifacts and their environments are becoming increasingly integrated. The user experience involves the offline as well as the online experience and spans over digital, physical, and service-based interactions.
These interactions need to be driven by data and a focus on the user. Try to understand how they made the design decisions that they made, instead of just assuming they used “best practice,” which is too vague of a justification for crucial design decisions. For example, the application is to provide remote operations monitoring of performance for a field workforce. When an asset is underperforming, a triangle with an exclamation mark appears next to the label of the asset. Why that shape? The key answer you are looking for here is that the iconography was already understood and in use by the end-users. This shows not only is the design team coordinating with a research team, but also that the designers make their decisions based on user needs, rather than their own judgment of what is aesthetically pleasing.
A key factor in the user-centered design process is the understanding of user needs. Think about the diverse set of users who will rely on the product you are considering creating. What will the different wants and needs of those users be? For example, you are creating a product to deploy all of your sales materials to channel sales reps. The marketing coordinator persona, responsible for updating the documents, needs to be able to easily delete older versions and replace with the most recent versions, as well to view the document usage by different sales reps in order to adjust the messaging. The channel sales rep, on the other hand, wants to be able to group documents into sets and tag their own documents for easy searching. The channel sales rep manager also wants to view the analytics and activity of the sales reps, along with dashboards of sales rep usage of documents based on experience and stage in the sales funnel to ensure all sales reps are using the materials properly. It’s important to ask the design team to look through their portfolio and see what kind of personas are driving the interactions. Find a small detail in their application and ask them to explain.
The next step is to ask about data, and how that factors into the design decisions. Think back to the Business KPIs section. How is the progress on meeting those factoring into each interaction? At each stage of design, is the progress toward meeting those goals being adequately measured? One practice to watch out for: designers who are improvised researchers. Research and design are very distinct practices that, while they work closely together, do require different backgrounds and specialties. When you request the background and duties of each member of the proposed design team, make sure that their duties are separate from those of the research team. The designers should communicate with you and your stakeholders, but doing field research, unless they have received specific training and a background (at least a graduate degree in psychology), then they are not qualified to conduct research.
Questions to include in the RFP for design:
- Please provide a resume, background and duties of the head of design and at least three senior members of the team. How much experience does the team have in your industry?
- Provide a walk through of the design process from start to finish for a mobile app. Look out for how many touchpoints the team will have with you and how frequently user feedback is incorporated into the project.
- Submit a portfolio of your work.
- For one project in the work, please submit the user personas that informed the creation of the interactions. Look for details in the persona that translated to the app.
- Explain how the personas needed specific changes that were incorporated into the final product. Find out how the team iterates on the initial concept and includes the personas into each stage of the process.
- How does design factor into the ROI of your projects? Are they thinking about the overall return on the project? Or are they just thinking “pretty?”
- How does your team design differently for a tablet vs. a phone vs. a watch? How do the interactions change for each medium? How well are they able to understand the design and user constraints for each medium? The screens are smaller, which is obvious, but user needs also change.
How to Get Mobile App Development Requirements Right— The First Time
In 1961, Alan Shepard piloted the mission into space. When reporters questioned him later on what he thought about as he waited for liftoff, he famously replied, “That every part of the ship was built by the lowest bidder.”
This sentiment echoes the practices of many companies in evaluating development teams with such high-stakes digital products. Outsourced teams with rock-bottom development prices seems like an easy solution to ease the budget, but what many miss in structuring a team this way is the creativity that experienced development teams bring to the development process. Evaluation processes need to not only evaluate how equipped the team is to handle product development, but also the intangible element of how well they can apply creative solutions. Part of that stems from working side-by-side with designers on the development in a product team, where both can benefit from brainstorming.
Dedicated product teams are the most effective configuration, for both creativity and understanding of your mobile product. Your product will have dedicated developers and designers working on it, and are able to use their deep understanding of the context of your product. Think about chefs working in a kitchen: each chef preparing one meal for table will create a better experience for the customer than a team of different chefs rotating in and out for each step, which is how some agencies approach development teams. A master to-do list dictates which tasks are upcoming for all of the products, and developers pick them up as they finish others. It’s easier on the agency, but the product loses that cohesive team.
Security is a major concern for companies pursuing digital products. The possibility of hacks of customer or company data and the million-dollar damage that comes with it is enough to dissuade any management team from investing in digital transformation. Experienced development teams can help to outline the data encryption and other precautionary measures required for a specific digital product. Security solutions need to be customized to the needs of the product, and experience in doing this needs to be included in any RFP.
In order to identify vendors with the right experience, you need to outline the back-end integrations, platforms and devices applicable to the product. Even if this is not set because you are looking to the development team for guidance, you should still provide the universe of possibilities so the vendor can insure that the resources are available.
A common resource problem is API shepherding. Too often, when writing the requirements for a mobile product, the manager of the product assumes the APIs are ready and functioning. By the time development launches, however, the team realizes this is not the case and thus, causes significant delays and cost overruns. Documenting APIs is also absolutely necessary for the experience of the development team and ability of the API to perform. Numerous tools exist to automate these tasks. Swagger, Mashery I/O Docs, and Apiary.IO are a few. You should provide a minimum level of documentation to evaluate an API’s ability to meet design requirements and build a necessary fake API server for development if needed. Your documentation should include a version history to note changes to APIs. It should also include a section outlining any policies and authentication requirements.
The final portion of your development section should address the methodology the development team follows for developing mobile applications. Not all methodologies are created equal; some methodologies will be more effective for your product and team. Consider how often you want to be communicated with and how much tweaking you will want to do based on user feedback.
Agile vs. Waterfall Development for Minimizing Risk
Agile software development focuses on iterative builds of small, client-approved portions that are delivered as soon as they are ready. Variations include scrum, XP, and DSDM.
- Regular, iterative releases for new features
- Continuous user feedback
- Constant communication with clients
- Dedicated product teams
Waterfall software development is a sequential design process, where each step is completed one after the other with meticulous documentation. The project is delivered all at once.
- Focus on record-keeping
- Lowers impact of development employee turnover
- Long development cycle
- Release of the entire product all at once
- User feedback after launch
Questions to include in the RFP:
- What type of development methodology does your team follow?
- Explain the major steps of your process of developing an app from start to finish. Watch out for long development cycles with no user feedback and a lack of incorporation of the design team.
- How will your team minimize risk? Some firms will give relatively standard answers here. Look for ones that include the business case in their development.
- Provide three examples where the development team applied creativity to a development issue.
- [If needed] Does your team have the capability to provide API shepherding as part of the project?
- Will you provide a dedicated product team who handles all tasks for the application? How will the team be configured?
- How often and well does your team communicate with clients?
- Explain how you would approach the security of this application.
Understanding Your Maintenance Needs
Creating a digital product, rather than completing a project, also creates another wrinkle for traditional RFP creation. Digital products are never finished, but projects are completed. This mirrors the reality of applications, who require regular maintenance and new version releases to address issues. Case in point: Throughout 2014 Amazon, Walmart, and Geico updated their respective apps between 20 and 25 times, and earned the highest average ratings in the App Store.
It’s important that the vendor is able to create a plan for each release, with analytics and measurement ongoing after the first release to improve the later iterations. The analytics and measurement should stem from the business KPIs outlined, as well as general usability and performance of the application.
Apple releases a new version of its OS each year, and at the very least, the application needs to conform to version specifications to remain functional. There are also new features with the releases that can make an impact with the user experience, such as the heightened sensitivity of the 3D touch that allowed for different actions based on the amount of force in the touch.
- How will an ongoing engagement for maintenance of the application work, in terms of pricing and labor? What type of maintenance plan can you provide?
- How will the performance of the app be measured after the launch through analytics? Will someone at the agency monitor and collect this data?
- What analytics do you recommend? What tools does your team use to monitor performance?
- Provide two examples of projects with ongoing maintenance and analytics, specifically describing how the app evolves with each release and why the specific analytics tools were selected.
Crafting Your Quality Assurance Strategy
Showing the timeline to the management team immediately turns into a discussion of where can cuts be made. Too often, this area is quality assurance of the application. Hershey’s made this mistake during their legacy ERP modernization– and saw $100 million worth of orders go unfulfilled.
Every RFP needs to assess the QA capabilities of agencies, who are guilty of the same mindset when it comes to QA. It’s cheaper to not test on real devices by using simulator programs, but these miss crucial bugs. The same applies for solely relying on automated testing, as opposed to including manual testing in the process as well. Automated testing is a great predictor of software behavior, not human behavior. You may be gambling on the quality of the product and risk botching the launch.
Questions to include in the RFP:
- Explain your QA process. Make sure that there is a thorough and rigorous process in place. Does the QA team learn about the process through their own experience?
- Does your QA team and development team work closely together?
- How many full-time employees are involved in QA? While it may not be completely necessary to use only FTEs, relying on all subcontractors poses risk.
- Does your QA Team test manually, automated or both? Do they use any simulator programs?