Clean drinking water and fisheries. These are all classic examples of the “tragedy of the commons,” the problem in economics and ecology where an individual is incentivized to consume as much of a shared resource as possible, but this comes at the expense of a larger society. With fisheries, a logical fisherman would like to make as many catches as possible. However, this can ruin the environment for everyone, as overfishing can endanger an entire species and even threatens whole ecosystems. The modern IT environment has given us a new case to consider related to the tragedy of the commons: data.
Data, much like drinking water, is a critical resource. Developers need to use this data, and logically, they want to use it as much as possible as it enables them to build innovative applications and products. However, too much data flowing around a modern distributed enterprise environment carries risks to the larger company.
The data developers are using often includes customers’ personal data like names and birth dates. The risks of this information being damaged or stolen can be catastrophic. This is where database administrators (DBAs), who are tasked with managing and securing this data, come in. There is an inherent tension between developers, who want as much access as possible, and the DBAs that want to limit that access. How data is used within an enterprise is critical because it must be used to create new solutions and be kept secure. An overly open data policy could lead to breaches and leaks, while an overly restrictive policy limits innovation.
Thankfully, there is a solution that is helping ease this tension. Observability can automatically analyze massive amounts of information across a business’s entire IT environment. Users can use it to pinpoint the causes of outages or performance issues and receive actionable insights to resolve these problems quickly. Modern observability solutions even use AI to predict and proactively prevent problems before they occur. DBAs and developers can utilize observability to strike a balance while boosting innovation, and improving productivity. Here are the three biggest ways that observability can solve the tragedy of the data commons and ease the feud between DBAs and developers.
Ensuring data is secure
One of the biggest concerns that DBAs have regarding data access is security. Sensitive production data are no longer stored exclusively on a company’s on-premise servers, where DBAs have full visibility into how it is being used and who has access. Instead, it is now stored across various private and public clouds as well as on-premises, which has reduced visibility for DBAs.
This sensitive data is the same information that developers often need to build and launch applications in a continuous delivery model, where developers build software in short cycles so it can be released at any time. This involves working across different clouds, including AWS and Azure, while leveraging complicated cloud-enabled technologies like microservices and Docker containers. When data is deployed across multiple clouds, the attack surface increases while visibility decreases, leading to new security concerns.
Observability provides DBAs with a better understanding of how data is being used by developer teams so they can more easily ensure that the data remains secure and protected. Additionally, many modern IT environments deploy threat intelligence solutions that observe actions and detect threat signals to help predict security incidents and respond before they happen. Some threat intelligence applications are powered by AI, which can automatically gather and organize the complex log data from the entire network. This data is then compared with an out-of-the-box threat database, helping to detect suspicious activity early. Observability and threat intelligence working in tangent provide a comprehensive overview to help relieve some of the security concerns for DBAs and create a more secure environment for data to be shared between teams.
Remaining data compliant
Ensuring that data is compliant is top of mind for every organization. Compliance regulations such as GDPR, CCPA, and HIPAA carry severe penalties for data leaks, meaning that companies must maintain the integrity and availability of sensitive and regulated data.
Ensuring compliance while managing requests for information from developers working across geographies and clouds can be a daunting task for DBAs. Not only must they remain compliant, but many businesses must demonstrate continuous compliance, which is only possible using complex, dynamic, and automated systems.
Observability allows teams to clearly see and follow the complex flow of data as it moves between DBAs, developers, and other teams. Additionally, observability can be deployed to help ensure that any sensitive data that is required to be anonymized is done so before it is shared. DBAs can further use observability to track the lineage of data as it moves between teams to monitor that it is not changed or transformed in any way that risk a compliance violation. Making compliance a regular part of the IT environment should not be viewed as a burden. In fact, it provides many benefits, including improving data quality, which can be used to further innovate and build better products.
Making teams more productive
Observability has shown it can create harmony between DBAs and developers by helping mitigate security and compliance concerns. Not only is it addressing these issues, but it is being used to make each team more productive on their own.
For DBAs, this means gaining additional insights that help ensure applications and services are running properly. Observability collects information from across hybrid and multi-cloud environments and presents it in a single view to allow DBAs to respond quickly and resolve issues. When DBAs gain the insights that observability provides, they can better manage critical data and anticipate future problems before they arise.
Observability is key to helping developers build applications as productively as possible. The ongoing “shift left,” has necessitated the use of observability as a way to gain single-pane-of-glass visibility into the build process. Developers are also provided insights into the performance of web applications and can move quickly to address any problems. Finally, Observability is improving the speed at which teams can innovate as it provides live code profiling that can be used to predict and prevent future problems.
Observability provides a real solution to ensure that data is not a resource that falls victim to the tragedy of commons, where its use can harm an entire organization. Instead, it is powering teams to use data securely to work more productively and build innovative applications.
Cullen Childress is Group Vice President, Product at SolarWinds.
TechNode Global INSIDER publishes contributions relevant to entrepreneurship and innovation. You may submit your own original or published contributions subject to editorial discretion.