VANCOUVER — A federal regional-development agency that spent more than $4 billion over a decade to support businesses in Western Canada did a poor job evaluating whether the companies it funded would be successful, according to an unpublished internal report.

“The market assessments conducted by [Western Economic Diversification] do not align with the degree of success of … clients,” reads a summary report of the agency’s economic impact, which The Logic obtained through an access-to-information request.

The regional development agency at the time known as Western Economic Diversification (WD)—the federal government has since split it in two, with Pacific Economic Development Canada (PacifiCan) responsible for B.C., and Prairies Economic Development Canada (PrairiesCan) for Alberta, Saskatchewan and Manitoba—hired independent evaluators in its 2019–20 fiscal year for the first-ever longitudinal study looking at the impact of its funding. The study, conducted by Vancouver-based Ference & Company Consulting, looked “to gather and analyze longer-term economic performance data of WD-assisted recipients,” according to the report, as the agency doesn’t require its clients to provide that information once its funding ends.

Between its 2009–10 and 2019–20 fiscal years, WD distributed more than $4 billion in funding to for-profit companies and non-profits in Western Canada, according to the report, which focused on two programs, including the Western Innovation Initiative (WINN), which loaned money to small- and medium-sized businesses in Western Canada to help them commercialize new technologies. 

The study included 50 of the first WINN clients that received assistance between 2014 and 2016 that accounted for 30 per cent of all of WINN’s funding, choosing a representative sample based on criteria including location and sector. One declined to participate, making the sample size 49.

It classified 28 of those projects, or 57 per cent, as “successful,” based on factors including commercialization of technology, profitability, job creation, as well as the company’s ability to raise more funds and to repay the WINN loan. It classified 11 projects as “somewhat successful” and 10 as “unsuccessful,” though it noted that some of the “somewhat successful” projects could eventually be reclassified as “unsuccessful,” as it was too early to determine whether they’d generate significant revenue.

Before it funded a project, WD would analyze its chances of success based on an assessment of the market, as well as the company’s finances, management, technology and the risks it faced; it would then combine those scores to arrive at an overall rating. 

The report found that WD’s market-assessment ratings consistently fell short. About 90 per cent of the projects that the report deemed unsuccessful had received a “very strong” market-assessment rating, as did 64 per cent of somewhat successful recipients and 71 per cent of successful ones.

PacifiCan and PrariesCan did not respond to specific questions from The Logic, but sent a statement saying the study was the first of its kind for WD and was approved this August. The agencies “will look at ways to implement best practices and lessons learned,” wrote Mark Feldbauer, spokesperson for PrairiesCan.

The report found that poor market research and testing was the primary reason WINN clients failed, and that the regional development agency could make its market assessments more accurate by requiring applicants to undertake primary market research before considering their applications, rather than relying on data from secondary sources. It could give applicants separate funding to do that market research. The report also proposed that WD work with independent marketing consultants for the market assessments, similar to how it was already working with the National Research Council of Canada’s Industrial Research Assistance Program to do its technological assessments. The partnership had made the technology assessments more accurate. 

The RDA’s assessments of applicants’ finances and management could also be improved, the report found. Nearly one-quarter of 13 clients that defaulted or were likely to default on their repayments attributed part of their failures to poor financial condition when they applied for funding. “It is critical that the financial assessments include” a company’s profitability, accumulated deficit and working capital available at the time of application, as well as whether it can cover projected losses, the report reads. Meanwhile, the study found that “almost all” approved applications received a “very strong” management-assessment rating, including those that were ultimately unsuccessful. Not one received a “fair” rating. It recommended that the criteria for that assessment be expanded to include whether the application accounted for risk factors and the likelihood of securing any necessary regulatory approvals, among other things.

The report confirms the value of these kinds of studies, PrairiesCan’s Feldbauer said, “very few” of which exist in the government sector. These can help departments “gain a more accurate perspective of the impact of assistance on funding recipients.”

For the two agencies, “the report provides valuable data to enhance the project-assessment process used to determine whether funding should be provided,” Feldbauer wrote. For example, it suggests requiring realistic revenue targets. The agencies will study the report and find ways to improve market-assessment accuracy, he said. They will also use the findings to “ultimately improve investor value for taxpayers.”

If WD had been better at picking winners, it might have seen a better return on its investment. The 49 projects, plus some second projects undertaken by nine WINN clients, resulted in $165.1 million in incremental revenue in 2019. All 130 WINN clients approved between 2014 and 2020 brought in an estimated $961 million in cumulative revenues over six years, the report found, or the equivalent of $6.46 for every dollar in WINN funding. The sample clients created more than 850 full time-equivalent jobs in 2019, with all WINN projects estimated to have created 1,905 jobs across the six years. That meant WINN spent an average of just over $78,000 per new job.

Still, overall, WINN clients outperformed against their peers that didn’t receive WD funding, according to the report. They were better able to reduce the growth of their debt, pay their employees more and grow their revenue.

Alberta had the fewest projects deemed successful among the four Western Canadian provinces, likely a result of trouble in the oil and gas industry. The most successful projects came from the cleantech and clean-energy sector followed by digital economy, new media, and information and communications technology, and then health and life sciences. 

The report also studied the impact of the Western Diversification Program, which made “strategic investments” in non-profit initiatives that would boost the region’s economy, according to the program’s website. However, “significant data limitations,” including a small sample size and a lack of data, complicated that analysis.