Brazil
Santa Catarina might be one of the smallest of Brazil’s 26 states but the region more than makes up for its size through its diverse and industrialized economy, which has the third-largest growth in the country, 2 percent above the national average. Santa Catarina’s Informatics and Automation Center (Centro de Informática e Automação de Santa Catarina, or CIASC) is a state-owned IT company that provides support and process-improving technology for the Santa Catarina government’s various departments.
Big data, but difficulty seeing the bigger picture
In the public sector, effective data sharing and easy access to information are a strong recipe for successful policy, yet the work culture is often resistant to inter-agency cooperation. In Brazil, CIASC sought to change that mindset by building an infrastructure for the gathering and standardization of data, accessible to all the state’s dozens of government departments.
Brazil’s political structure prescribes a system of shared responsibility among its federate entities (federal government, states, and regional counties). In this system health care, law enforcement, education, and almost every other essential service relies on the state to some extent. One of CIASC’s most important roles is developing data-driven technical solutions in support of the state policies governing the departments that deliver those services.
Unfortunately, Santa Catarina’s government workers were not consistently sharing either data or knowledge, a disconnect not uncommon in public sector entities with tight resources. One department would create a process for using data, and another with a similar need would create its own process. For example, if someone in the agricultural department requested administrative information from the state’s human resources department, agriculture would receive a data dump from HR’s data warehouse rather than being given access to usable information from the start. The next department needing that information would submit its own request and receive a similar data download. The difficulty inherent in the process discouraged information sharing and hindered the government from making data-based decisions when forming public policy.
The government needed a holistic platform that could help its administrators, IT staff and policymakers make sense of data from different departments and see the bigger picture of residents’ needs across the state.
One state, one data infrastructure
In 2016, CIASC began working with Cloudera to develop a data infrastructure that would support Santa Catarina’s regulatory efforts to encourage digital transformation and inter-agency data sharing. A small team of the company’s technicians and managers designed a system for the state secretary of finance to improve the performance of electronic invoices processing. The massive amount of data used to generate these invoices required the team to consider a new approach to data management.
Within four weeks, CIASC devised and deployed a distributed data processing infrastructure using an open framework that could efficiently store and process large datasets. In some cases, the new system enabled the state’s data managers to access information 14 times faster than they could previously. The more datasets could be integrated into the project, the more public officials could rely on the infrastructure to help them solve complex problems. Beyond improving invoice processing, the framework’s success helped change the public sector’s mindset by showing the value of open access to data.
Later called the BoaVista Project, the infrastructure offered a centralized system to meet the needs of all the state’s departments. To encourage departmental workers to use the new system, CIASC created a standard login and standards for data ingested into the framework. The company later offered training courses to help departmental workers create database queries within the framework that they could use and share with colleagues in other areas of the state government.
Data integration leading to bigger and better services
BoaVista helped change the way people think about sharing information throughout the Santa Catarina government. The infrastructure was particularly useful during the early days of the COVID-19 pandemic, when government agencies were required to quickly gather and communicate information with the federal government. With BoaVista, CIASC had already put the foundation of such a data sharing system in place.
Every major departmental secretary – including finance, health, security, and agriculture – interacts with the BoaVista Platform, as do the state’s Court and Public Prosecutors Office. Some areas of the federal government likewise use Santa Catarina’s data-sharing infrastructure. The data-sharing infrastructure currently incorporates approximately 140 datasets, more than 4,000 tables accessed by more than 1500 users in 35 different public entities across Santa Catarina’s state government. These public-sector users are responsible for more than 90% of Santa Catarina’s annual budget.
Such sharing enables government agencies to both make better use of their own data and to correlate that data with information generated by other government agencies. Farmers, agriculture technicians, and public officials, for example, rely on the InfoAgro platform (which BoaVista supports) for simplified access to dynamic, up-to-date information related to crop productivity in the state, agricultural costs and prices. Another example has been the Education Department’s ability to improve its human resources management by contrasting information in its systems with data provided by the state’s Administration Department, responsible for human resources.
After years of growing the data infrastructure’s capabilities and capacity, CIASC is migrating it to Cloudera to "empower governmental departments to develop rules-based access to their data, something CIASC had been doing for them,” said Raphael Tucunduva Gonçalves, CIASC’s Data Intelligence and Enterprise Solutions Manager.
“The move to Cloudera frees up CIASC to plan new ways of using machine learning and artificial intelligence. We have a very well-organized repository of data across the state that ML and AI could use to anticipate the needs of the state and its citizens. Those are the types of solutions we want to offer our departments in the future,” Tucunduva Gonçalves added. Examples might include access to prediction models that improve the productivity of corn crops and mechanisms to improve cybersecurity.
CIASC has been very pleased with Cloudera’s customer support throughout the transition. “With Cloudera’s customer support, we were able to meet our goals much faster than we would do without. It’s so much easier when we can consult with people who know the platform we use and have effective answers to our demands and challenges. This has increased significantly our capacity to deliver results and our confidence in sustaining such a complex platform,” Tucunduva Gonçalves said.
