Measuring Innovation

Innovation transforms pure science into relative economic and societal advantage around the world. Not just in countries, but in regions too. It defines today’s and tomorrow’s cities of success.

So how do you measure innovation?

  • Is it by the number of patents for new ideas a country or region registers with international gatekeepers?
  • Is it by the raw number of scientific papers published by a country or region’s top universities, whether online or in prestigious journals?
  • Or is it by ‘creative energy’, as measured by the growth in scientific production in any given region?

The right answer makes a huge difference in a country such as Brazil. Yet there is no right answer.

Measure by the first yardstick – the number of patents registered – and the country is a disappointing laggard. Measure by the second yardstick – scientific papers – and parts of Brazil make a respectable showing (even if the bulk of papers are in Portuguese and so have limited international influence). Measure by the third yardstick and one region – São Paulo – is a world-beater, grouped alongside Seoul, Shanghai and Beijing.

Let’s analyse the way innovation rankings are produced and the quality of their data to see whether or not the scales are tipped against Latin American nations.

  1. Patent Registration by Domicile

The most conservative measure of innovation is controlled by Geneva-based World Intellectual Property Organisation (WIPO), a self-funding agency of the UN with 189 member states that since 1967 has been selling its services, and which recruits management consultancies and top universities to enhance its prestige. WIPO has access to all the top patent databases and is the appointed guardian and guarantor of intellectual property empowered to resolve disputes.

WIPO’s latest research analysed the addresses of inventors named in all 950,000 international patent applications published between 2011 and 2015 under the Patent Cooperation Treaty. Its Global Innovation Index 2017 contains two important rankings. The first is nation-by-nation, and the second is measure by region or innovation cluster, without regard for national boundary.

WIPO's Innovation survey cover.

WIPO’s Innovation survey cover.

According to WIPO rankings, Latin America’s economic giant Brazil trails behind even its smaller regional neighbours, while East Asian nations boasting the world-beating clusters of Japan’s Tokyo-Yokohama and China’s Shenzhen-Hong Kong are far over the horizon.

Rankings in The Global Innovation Index 2017 place Brazil in 69th out of 127 places – behind Mexico, Columbia, Uruguay, Chile and even Costa Rica. Its BRIC partners India, Russia and China are all much further up the list, topped by Switzerland, Sweden and the Netherlands.

And in the report’s special section Identifying and Ranking the World’s Largest Clusters of Inventive Activity, no Latin American region makes the top 100.

How could Brazil’s economic might (it is a G20 nation with a GDP of US$1.79 trillion, almost three times Argentina’s and nearly double Mexico’s) have translated into such a poor showing in the innovation stakes when measured nation-by-nation?

The answer becomes more nuanced when we view innovation not only in national terms, but using the same regional yardstick adopted elsewhere by WIPO. The findings of WIPO special section overlay national analysis with study of regional hotspots, to generate a nuanced answer globally.

Japan may be ranked just 14 on WIPO’s tally of nations, yet Tokyo-Yokohama is the world’s largest innovation cluster while two others, Osaka-Kobe-Kyoto and Nagoya, are also in the global top ten. Likewise China, which ranks 22 on a national scale, has the world’s second largest cluster in Shenzhen-Hong Kong. By contrast the US, which ranks fourth on a national scale, has Silicon Valley which is the world’s third largest such cluster. In fact the US has 32 of the world’s 100 top clusters.

There’s more. Why, for instance, does WIPO place Switzerland at the very top of its innovation rankings?

Not, surely, for the quality of original thought by just eight million Swiss nationals or its universities, however prestigious. Swiss-registered pharmaceutical companies have the money and power to buy the best scientific knowhow from around the world and register these discoveries as “made in Switzerland where the patent laws and stock markets are strong,” thus ensuring that country occupies the number one spot. So WIPO’s is a “who owns what” ranking, and not a “where do they have the best original ideas” ranking.

Ironically given the country’s poor showing, publication of the 2017 Global Innovation Index was partly funded by Brazil’s federal agencies CNI, SESI, SENAI and Sebrae. (These are respectively an industry club and training and small business support quangos financed by blanket taxes on industry.) A table in the WIPO  report  shows how much Brazil’s federal government spent on innovation subsidies in 2012 alone.

Table  with Brazilian  data from PF Zunga published  by WIPO with report from  CNI/SESI/SENAI/Sebrae

Table with Brazilian data from PF Zunga published by WIPO with report from CNI/SESI/SENAI/Sebrae ( In 2012 exchange rate was BR$2.04 = 1 USD)

This earned them a special chapter on Policies and Institutions Fostering Innovation and Agriculture Technologies in Brazil. Signed off by Robson Braga de Andrade and Guilherme Afif Domingos, (respectively the presidents of CNI/SENAI and Sebrae) the authors acknowledge that Brazil “faces many challenges in fostering science and technology and creating an environment more suitable for innovation.”

The authors describe agriculture and agricultural research as a beacon of innovation, highlighting the complex web of federal regulations on industry with tax incentives and subsidized credits for R&D. The authors cite the existence of sectoral funds for biotechnology and agribusiness funded from the public sector plus a welter of legislation starting with the 1996 Industrial Property Law, the 2004 Innovation Law and the 2005 Incentive Law, as meaning that Brazil now has “many of the instruments used in the developed world to foster innovation.”

Agribusiness sector is  Brazil's innovation poster child

Agribusiness sector is Brazil’s innovation poster child

So why then, despite all this huge effort and expense, does Brazil perform so poorly with its 69th ranked WIPO place? Either Brazil is doing something wrong – or it’s measuring something wrong.

Well, Brazil does have regional innovation hotspots and in relative terms, these are performing well. But they won’t show up on the WIPO radar. This is because few inventors would choose to register their patents where their only protection would be Brazil’s creaking legal system, and despite the presence of federal laws there is only a fledgling infrastructure of private sector legal and investor support for innovators who would become entrepreneurs.

So to pinpoint Brazilian innovation, we need to look beyond the narrow legal basis of the “who owns what” WIPO rankings to a different yardstick.

  1. Publication of Scientific Papers.

Every academic knows the “publish or perish” rule and in recent years the global volume of papers has soared. So many papers are published that impact – rather than total input – is now a more reliable measurement of true innovation activity. So the number of citations in other international papers is more important than raw output.

This data has begun to be organized in ways that locate those geographies where original thought is coming from. In June 2015 Méric Gertler, a professor in the Department of Geography and Planning and currently chancellor of the University of Toronto, Canada, presented an important paper at the Glion Colloquium in Switzerland. This is a biennial forum attended by leaders of research universities.

Gertler produced some startling statistics about innovation cities and clusters that contrast markedly with the WIPO rankings. There are described in detail in this 2015 article which explains his methodology – and the results which look quite favourable to Brazil.

Research by M Gertler originally published by  FAPESP Pesquisa

Research by M Gertler originally published by FAPESP Pesquisa

Gertler compiled data on scientific production in regional clusters, extracted from the Web of Science, a Thomson Reuters website, and analyzed networks of collaborations connected with that production.

This generated his first set of rankings (in the illustration with orange captions on the left) which show – predictably enough – that those traditional centres of academic research and learning that are also centres of international academic publishing, are the standout leaders. So London (1st), Boston, Tokyo and Amsterdam are predictably in the top 10. So too are Beijing (2nd) and Seoul (7th).

While the data shows a preponderance of North American and European cities as top innovation centres, it also shows São Paulo in 32nd position, with about 40,000 publications. This is behind more consolidated regions like San Francisco, Tokyo, and Berlin but ahead of major regions like Munich and Manchester-Liverpool.

One reason for this is São Paulo’s support for SciELO (Scientific Electronic Library Online), a made-in-Brazil open-access online platform for sharing academic papers that has revolutionized access for academics outside the charmed circle of for-profit academic journals controlled by US, UK and Dutch companies. SciELO is product of a partnership among FAPESP ( – the State of São Paulo Science Foundation, BIREME ( – the Latin America and Caribbean Center on Health Sciences Information, as well as national and international institutions related to scientific communication and editors. It publishes in English, Spanish and Portuguese. And it has a linkup with the Web of Science.

Brazil-based  online Library of Science is altering  global publishing trends

Brazil-based and free access, this Library of Science is altering global publishing trends.

Thanks in part to SciELO, Brazil has been at the forefront of the Open Access revolution for free publication of scientific data.  In fact it has been described as “the world’s most open” research.  In conjunction with US end EU lawmakers imposing similar rules for the availability of any research fully or partly funded from public sources. You can read an earlier SFB article about the evolution of “Green/Gold” scientific publishing rules by clicking here. You can read about the growth in SciELO’s English language index publications (which in 2014 overtook Spanish and Portuguese) by clicking here.

Most importantly, Gertler’s findings based on publication of research data show not only the static position of innovation centres around the world, but their relative growth.

This not only helps to identify growing hotspots or innovation clusters, but also helps to define the third category: ‘creative energy’, as measured by the growth in scientific production in any given region

  1. ‘Creative Energy” in Scientific Production

Gertler’s data allows researchers to compare the changing performance of each region on his list between 1996 and 2013. Regions with especially strong growth will display a burst of publishing activity that – arguably – serves as a proxy for later innovation as pure science meets practical application.

According to writer Fabrício Marques of Pesquisa FAPESP, Gertler’s analysis gets closer to mapping how research universities imprint a dynamic spirit on their home regions, jump-starting the economy, innovation, and creativity. In other words, this could be a more reliable methodology for identifying the “secret sauce” that makes some innovation clusters successful, than simply logging patent registration as an index of commercial activity.

São Paulo economic powerhouse filling  gaps in federal funding.

São Paulo economic powerhouse filling gaps in federal funding.

In Gertler’s ranking (in the illustration with green captions on the right), which illustrates the evolution of scientific output in the last two decades, São Paulo (with an increase of more than 400%) and the Asian metropolises (Shanghai grew by 1000%) appear at the top. By contrast, mature scientific centres like Munich, in Germany, Boston, in the United States, and London, experience much more muted growth in publication volume.

That’s not to say innovation doesn’t happen in big, established cities. Gertler also showed that among the 50 highest-regarded universities on the Times Higher Education World University Rankings, only seven are situated in urban centers that have a population of less than one million.

In the other 43, the presence of a world-class university is always tied to some major metropolitan region where companies and institutions benefit from the knowledge and human resources generated by academia while at the same time making demands that challenge the academic community.

The São Paulo urban cluster is defined, in Gertler’s analysis, as the megalopolis of more than 30 million residents formed by the state capital and the cities of Campinas and São José dos Campos, where there are institutions like the University of São Paulo (USP) and the University of Campinas (Unicamp), the federal universities of the ABC region (UFABC) and of São Paulo (Unifesp), and the Technological Institute of Aeronautics (ITA), as well as three institutions that are part of São Paulo State University (Unesp).

Campinas University   is part of the São Paulo  innovation cluster

Campinas University is part of the São Paulo innovation cluster

USP, which whose main campus is in the state capital city, is itself responsible for 22% of Brazilian scientific output, according to figures from the Web of Science. “That Brazilian hub is part of a remarkable story. As recently as 25 years ago, it was little known and highly specialized. Now it is a global force,” Méric Gertler told Pesquisa FAPESP. “According to bibliometric data from Thomson Reuters, in 1990 USP was collaborating with 350 institutions in 28 countries. By 2014, it was working with more than 6,500 institutions in 145 countries. That is simply extraordinary.”

Fabrício Marques of Pesquisa FAPESP uncovered a second, earlier innovation ranking for world cities based on comparable “bibliometric” or science publishing yardstick to Gertler’s. Christian Wichmann Matthiessen from the University of Copenhagen, together with Annette Winkel Schwarz and Soren Find from the Technical University of Denmark produced a similar study. (click here). They warn that bibliometric indicators say a lot about quantity but not necessarily much about quality of thought.

What’s clear is that innovation and the mix of ingredients necessary foster a true innovation cluster in some cities, is by no means subject to empirical measurement.

São Paulo's  research publication output is one proxy for innovation.

São Paulo’s research publication output is one proxy for innovation.

While the lowest common denominator of domicile of patent registrations may be too  blunt an instrument to provide an accurate portrait of what is going on in the world, it is equally naïve to believe that gross volumes of academic publishing are anything more than the roughest of proxies for eventual industrial and commercial activity.

Likewise – as Brazil’s case attests so well – a plethora of federally-mandated rules, regulations and subsidy packages don’t themselves make great ideas, or even guarantee that great ideas will ever become great products or solutions.

Researching in Brazil

Brazil’s best-known scientist, the veteran nuclear physicist  José Goldemberg,  in September 2015 took office as the president of  the São…

Read more