## **How Geospatial Data is Changing the Future of Databases**

Geospatial data has become essential for modern businesses, driving efficiency in logistics, retail, finance, and urban planning. As mobile devices, IoT sensors, and real-time tracking generate vast amounts of location-based data, businesses must adapt to leverage its potential.

Unlike traditional data, geospatial information is **relational and dynamic**, requiring specialized indexing and processing. Many industries rely on it for:

*   **Retail & E-commerce**: Delivery optimization and personalized customer experiences.
*   **Logistics & Transportation**: Route planning and fleet management.
*   **Finance & Security**: Fraud detection through geospatial anomaly analysis.
*   **Smart Cities & Urban Planning**: Infrastructure development and environmental monitoring.

Executives must recognize when geospatial capabilities become critical to their technology stack. This article explores what makes geospatial data unique, when to prioritize it, and how modern databases handle location-based information efficiently.  

## **What Makes Geospatial Data Unique?**

If you expected to first read about the technical complexities of points, lines, and polygons, I want to prioritize what I consider a real (and often neglected) aspect.

### **1️⃣ The Dynamic Nature of Geospatial Data**

Unlike static business records, geospatial data is constantly **changing and evolving**:

*   **Real-time Movement**: Vehicles, people, and shipments move dynamically, requiring databases to update positions frequently.
*   **Changing Boundaries**: Administrative boundaries, roads, and zoning laws can shift over time, affecting spatial queries.
*   **Context Awareness**: Some geospatial queries are **context-dependent**, meaning the same query might return different results based on external factors like traffic, weather, or time of day.

These unique characteristics of geospatial data **require specialized database architectures** capable of handling **spatial indexing, real-time updates, and multidimensional relationships** efficiently.

### **2️⃣ Geospatial Data is More Than Just Coordinates**

At its core, geospatial data consists of **latitude and longitude points**, but it is much richer than simple Cartesian coordinates. Real-world applications often involve:

*   **Points** (specific locations, such as a store or a Wi-Fi access point)
*   **Lines** (road networks, pipelines, or flight paths)
*   **Polygons** (city boundaries, zip codes, and regions affected by a weather event)
*   **Time-dependent Data** (tracking movement over time, such as vehicle GPS data or migrating wildlife)

These elements introduce **spatial relationships**, such as **distance, adjacency, containment, and overlap**, which are crucial for applications like route planning, proximity searches, and environmental monitoring.

### **3️⃣ Traditional Databases Struggle with Geospatial Data**

Traditional relational databases (RDBMS) were designed to work with structured data in tables, where queries involve **simple key-value lookups** or **aggregations**. However, geospatial data requires:

*   **Complex Spatial Queries**: Instead of simple comparisons (e.g., `WHERE city='New York'`), geospatial queries ask questions like:
*   “What are the **nearest** gas stations to my location?”
*   “Which delivery zones **overlap** with this zip code?”
*   “What customers are _**currently**_ within a **5-mile radius** of this point?”
*   **Massive Data Sets**: Maps, satellite imagery, GPS logs, and sensor data generate large volumes of information that require efficient storage and retrieval.
*   **Real-time Processing**: Many applications (e.g., ride-sharing, traffic monitoring) require **instant geospatial lookups**, which traditional databases are not optimized for.

### **4️⃣ The Need for Specialized Indexing**

To efficiently handle geospatial queries, databases use specialized indexing techniques that differ from traditional B-tree or hash-based indexes:

*   **R-Trees**: Hierarchical tree structures optimized for bounding-box searches, used in PostGIS and spatial databases.
*   **QuadTrees**: Recursive division of space into quadrants, commonly used for geospatial partitioning.
*   **Geohash**: A string-based encoding of latitude/longitude data that allows fast spatial queries with a compact representation, often used in in-memory databases like Redis.

![image (2).png](https://janea-minio.apps-us2.janeasystems.com/public/image_2_29080f6636.png)

Illustration of a PR Quadtree: Spatial Partitioning and Tree Structure Representation

Without these **geospatial-aware indexes**, even simple queries — like finding the nearest store — would require scanning millions of records, making them computationally expensive and slow.

## **Making Data-Driven Decisions with Geospatial Technology**

Understanding these complexities is crucial for executives deciding **when and how** to invest in geospatial technology. The next articles will delve into specific business cases where companies should prioritize geospatial data integration and the impact it can have on decision-making and operations.

---

_If you’d like to explore how your organization can effectively leverage geospatial data or if you have questions about implementing specialized geospatial technologies, feel free to_ [_contact us_](https://www.janeasystems.com/contact)_. We're here to help you navigate these complexities and uncover new opportunities for your business._

Geospatial data is dynamic and relational, making traditional databases inefficient. Learn why RDBMS struggles with geospatial queries and how specialized indexing (R-Trees, QuadTrees, Geohash) improves performance.

Why Traditional Databases Struggle with Geospatial Data & How to Fix It

Why Executives Should Care About Geospatial Data

Bring Redis, a modern, high-performance cache, to z/OS mainframes. Explore use cases, technical hurdles, and strategic benefits in this first look from Janea Systems.

Porting Redis to z/OS: Ambitious, but Worth It | Blog

Porting Redis to z/OS

## What Problems Do 3D Tiles Solve?

**Massive 3D scenes** — such as city models, infrastructure networks, and digital twins — are incredibly large and complex, often containing billions of polygons and vast amounts of data.

Traditional approaches require downloading entire datasets upfront, resulting in slow load times, excessive bandwidth use, and poor performance, especially on web and mobile devices.

Additionally, modern photogrammetry technologies (like **LiDAR**) offer highly detailed geometry and texture, which make it virtually impossible to load entire datasets in commodity hardware, laptops, and phones.

## How Streaming Solves Real-Time 3D Challenges

**3D Tiles** revolutionize massive 3D visualization by enabling dynamic streaming.

Instead of loading an entire dataset upfront, only the tiles visible in the user’s current view are streamed on demand. As users pan, zoom, or navigate, new tiles load progressively, while out-of-view files are unloaded, drastically reducing bandwidth and load times.

This streaming approach supports progressive refinement-low-resolution tiles appear first, followed by their higher-detail versions-ensuring smooth, real-time interaction even with enormous datasets. Optimized for web environments, streaming makes it possible to explore massive 3D scenes directly in browsers without heavy downloads or delays.

## How 3D Tiling Improves Large-Scale 3D Performance

3D Tiles break massive 3D scenes into smaller, spatially organized tiles arranged hierarchically. Each tile represents a manageable chunk of data with multiple levels of detail (LOD). Distant tiles load simplified geometry, while nearby tiles load detailed models, balancing performance and visual quality. This tiling enables efficient spatial culling, where only visible tiles are rendered, which reduces processing and GPU load. The approach supports diverse data types, including meshes, point clouds, and BIM models. Preprocessing pipelines optimize tiles through compression and spatial indexing, ensuring fast streaming and smooth visualization.

3D Tiles break massive 3D scenes into smaller, spatially organized tiles arranged hierarchically. Each tile represents a manageable chunk of data and supports multiple levels of detail (LOD). Distant tiles load simplified geometry, while nearby tiles load detailed models, balancing performance and visual quality. This tiling enables efficient spatial culling, where only visible tiles are rendered, which reduces processing and GPU load. The approach supports diverse data types, including meshes, point clouds, and BIM models. Preprocessing pipelines optimize tiles through compression and spatial indexing, ensuring fast streaming and smooth visualization.

![NoBanksInThisOne (1) (1) (1).gif](https://janea-minio.apps-us2.janeasystems.com/public/No_Banks_In_This_One_1_1_1_e44d355a98.gif)

## Under the Hood: 3D Tiles Internals

At its core, 3D Tiles uses a JSON-based tileset that describes the hierarchical structure and metadata of the scene. Tiles themselves contain binary data formats such as **b3dm** (batched 3D models), **i3dm** (instanced models), and **pnts** (point clouds). Each tile includes bounding volumes for efficient detail transitions. The format supports extensions for materials, lighting, and metadata, enabling rich and flexible visualizations. Tiles are typically hosted on web servers or CDNs, allowing fast, global access. Client-side engines like CesiumJS handle dynamic tile loading, caching, and rendering for a seamless user experience.

## CesiumJS and Cesium ion: The Ecosystem

[CesiumJS](https://cesium.com/) is a leading open-source 3D geospatial engine that pioneered support for 3D Tiles, enabling efficient streaming, level of detail management, and high-performance rendering directly in web browsers. Cesium ion is Cesium’s cloud platform that processes raw 3D data into optimized 3D Tiles, hosts tilesets on a global CDN, and provides easy tools for uploading, styling, and streaming massive 3D models. Together, CesiumJS and Cesium ion form a powerful ecosystem empowering developers to build scalable, interactive 3D applications with minimal infrastructure overhead. Continuous innovation and community support have made 3D Tiles widely accessible and practical for real-world projects.

## The Future of 3D Tiles in Geospatial Innovation

The future of 3D Tiles is bright as demand grows for real-time, high-fidelity 3D data in smart cities, AR/VR, autonomous systems, and digital twins. Ongoing advancements aim to improve streaming efficiency, tile compression, and support for richer materials and semantic metadata. Integration with AI and machine learning will enable smarter data processing and scene understanding. The open ecosystem continues to expand, fostering interoperability and innovation. Enhanced collaboration and real-time updates will make 3D Tiles even more powerful for dynamic, large-scale 3D applications. Ultimately, 3D Tiles will remain foundational to the next generation of immersive spatial computing.

---

Whether you're building immersive 3D experiences, streaming massive geospatial datasets, or deploying Cesium for real-world applications, success depends on a deep understanding of both the tools and the terrain. As a [certified Cesium partner, Janea Systems](https://cesium.com/certified-developer-directory/) brings together specialized geospatial engineering and proven software expertise to help you turn complex data into high-performance solutions.

Looking to push the boundaries of what’s possible with 3D geospatial technology? [Let’s talk](https://www.janeasystems.com/contact).

Discover how 3D Tiles and Cesium enable real-time streaming, tiling, and visualization of massive 3D geospatial data in web-based applications.

How 3D Tiles and CesiumJS Transform Massive 3D Data

Why 3D Tiles Change the Game for Massive 3D Scenes

Inserting and maintaining data in a relational database is expensive. Every write must update one or more **indexes** (data structures such as B-trees) that accelerate reads at the cost of extra CPU, memory, and I/O. On a single node, tables start to struggle once they pass a few terabytes. Distributed SQL and NoSQL systems push that limit, but the fundamental write amplification costs remain.

## Object Storage

To escape those costs, teams began landing raw data in cloud object stores like **Amazon** **S3**. Instead of hot indexes, query engines (Spark, Athena, Trino) rely on partition pruning and lightweight statistics. This led to dramatically lower storage bills and petabyte-scale datasets on commodity hardware.

## Vectors Embeddings

AI and LLM workloads now emit **vector embeddings –** hundreds or thousands of dimensions per record. Answering “_Which vectors are nearest to this one?”_ in real time is tricky:

*   High-dimensional data breaks classic data structures.
*   We lean on approximate nearest neighbor (ANN) algorithms such as **HNSW** or **IVFPQ**.
*   Queries often combine a distance threshold with metadata filters.
*   Recall, precision, and latency form a three-way tradeoff.

## Amazon S3 Vector: A Game-Changer

Announced [yesterday](https://aws.amazon.com/blogs/aws/introducing-amazon-s3-vectors-first-cloud-storage-with-native-vector-support-at-scale/), [**Amazon** **S3** **Vectors**](https://aws.amazon.com/s3/features/vectors/) brings vector-aware storage classes to S3. Each vector table:

*   Stores vectors of fixed dimensionality, compressed on write.
*   Supports ANN search plus filters on metadata.
*   Delivers sub-second latency: great for batch, a bit slow for interactive UX.

## Closing the Latency Gap with In-Memory Caching

Janea Systems’ background is deeply rooted in working with **in-memory**, low-latency caches. Our track record includes:

We are the creators of [Memurai,](https://www.memurai.com/) the official Redis for Windows, trusted by developers for its performance and reliability.

We are active contributors to [ValKey](https://valkey.io/), a rapidly evolving open-source fork of Redis, pushing the boundaries of in-memory data stores.

Given the inherent characteristics of S3 Vectors, including powerful storage and batch processing, but with room for improvement in interactive scenarios, the next logical step is to strategically implement a high-performance cache on top of S3 Vectors.

We are excited about the possibilities Amazon S3 Vectors unlocks. The upcoming articles will cover how to effectively integrate Redis, Valkey, or Memurai with the S3 Vector service to achieve optimal performance for your AI/LLM workloads. Also, we will explore the new AWS service and its implications for modern data architectures in detail. Stay tuned!

Explore how Amazon S3 Vectors change vector embeddings for AI/LLM workloads, offering cost-effective, petabyte-scale storage.

S3 Vectors: Changing How We Think About Vector Embeddings 

_...And Say Goodbye to Expensive SaaS Pricing._

Vector databases are powerful — and often expensive. In this article, you'll learn how to build a scalable, low-cost vector search system using S3 Vectors, Product Quantization (PQ), and open-source tools.

---

Here’s an estimated price comparison for storing 1 billion and 10 billion vectors using the most common SaaS vector databases. These numbers are pulled directly from each provider’s pricing calculator.

![https___dev-to-uploads.s3.amazonaws.com_uploads_articles_8exmvfv2yg4ydcx1w1cp.webp](https://janea-minio.apps-us2.janeasystems.com/public/https_dev_to_uploads_s3_amazonaws_com_uploads_articles_8exmvfv2yg4ydcx1w1cp_32d059c157.webp)

Want the short version?

> _Vector databases are expensive because they rely on powerful, always-on hardware. They keep in-memory indexes fresh and caches hot — and that costs money._

## **Old Tricks, New Vector World**

For tabular or log data, we decoupled compute and storage a long time ago: store data cheaply in S3 or MinIO, and spin up compute (like Spark or Presto) only when needed.

Amazon has now extended this model to vector embeddings with Amazon S3 Vectors. \[Quick dive [here](https://www.janeasystems.com/blog/s3-vectors-changing-how-we-think-about-vector-embeddings)\]

S3 Vectors lets you store huge volumes of vector data at low cost and run similarity searches in under a second — ideal for batch workloads and analytics.

_Can we ditch expensive Vector DBs now?_

Not quite. S3 Vectors doesn’t offer the low latency needed for real-time use cases — like fraud detection, recommendation engines, or chatbots that require sub-100 ms responses.

Instead, think of S3 Vectors as the durable, budget-friendly foundation. You’ll still need a lightweight layer on top to meet real-time latency requirements.

## **Level 1: Download and Run**

Let’s start simple: use open-source software out of the box, no code changes, just run it.

We lower similarity search latency by using a classic Computer Science technique — indexes — and storing data in RAM (which is fast but expensive).

## **Product Quantization (PQ): Fast, Memory-Efficient Search**

Performing exact distance calculations (cosine, Euclidean) on billions of 768-dimensional vectors is too slow and compute-heavy.

Product Quantization (PQ) helps by compressing vectors into compact forms. This makes searches 10–100× faster — with minimal accuracy loss.

### **How PQ Works**

PQ splits each high-dimensional vector into smaller chunks (e.g., groups of 8 dimensions), then maps each chunk to the closest centroid in a precomputed codebook. Only the centroid IDs are stored.

At query time, instead of comparing against billions of raw vectors, the system compares to ~256 centroids per chunk — massively reducing compute time.

For most NLP workloads, PQ delivers excellent recall while cutting memory and compute costs.

![https___dev-to-uploads.s3.amazonaws.com_uploads_articles_uh0tf0n5lzh33kx0ti9s.webp](https://janea-minio.apps-us2.janeasystems.com/public/https_dev_to_uploads_s3_amazonaws_com_uploads_articles_uh0tf0n5lzh33kx0ti9s_c1c1c73d3d.webp)

## **Tool Selection**

FAISS — originally developed by Facebook AI Research — is the go-to library for efficient similarity search and clustering of dense vectors. It’s widely adopted for high-performance vector indexing at scale. But I recommend [JECQ, a drop-in replacement with 6× lower memory usage.](https://www.janeasystems.com/blog/jecq-smart-open-source-compression-for-faiss-users)

_Disclaimer: I created JECQ. That said, it works well. But use FAISS if you prefer._

You can find [the open-source JECQ project on GitHub.](https://github.com/JaneaSystems/jecq)

## **In-Memory Cache**

You can cache a subset of raw vectors in RAM using tools like Redis or Valkey, depending on your licensing needs.

For 10 billion vectors (~30TB in S3), storing just 1% in RAM (about 300GB) can make a big difference. Pricey, but manageable.

## **Level 1 Architecture**

Let’s walk through the architecture:

![https___dev-to-uploads.s3.amazonaws.com_uploads_articles_9jsotg9tntv2grw9neg3.webp](https://janea-minio.apps-us2.janeasystems.com/public/https_dev_to_uploads_s3_amazonaws_com_uploads_articles_9jsotg9tntv2grw9neg3_1b447b7c46.webp)

*   A router service handles incoming similarity search requests. It’s stateless and can scale horizontally.
*   Each node loads a copy of the JECQ (or FAISS) index in memory.
*   The router uses JECQ to find candidate vector IDs.
*   It then checks Redis for raw vectors:
    *   **Cache hit:** Redis returns vectors. Router re-ranks and returns results.
    *   **Cache miss:** Router pulls vectors from S3, re-ranks, and returns results.

## **Level 2: Teaser**

Level 1 works fine for datasets up to ~1 billion vectors or demo workloads. But if you want 10–100 ms P95 latency at multi-billion scale, you’ll need more:

*   **Local raw vectors on NVMe:** A middle layer (5–10% of raw size, ~1.5TB) between RAM and S3 to avoid frequent S3 fetches.

![https___dev-to-uploads.s3.amazonaws.com_uploads_articles_rg0sihlvptis6zovtcce.webp](https://janea-minio.apps-us2.janeasystems.com/public/https_dev_to_uploads_s3_amazonaws_com_uploads_articles_rg0sihlvptis6zovtcce_cd9247645f.webp)

*   **Hierarchical data layers:** JECQ + Redis/NVMe integration enables local posting list retrieval, turning 100 ms S3 reads into 2–5 ms NVMe reads.
*   **Index sharding:** Splits PQ clusters across nodes and avoids duplicating 100GB+ compressed data per node.
*   **Advanced cache management:** Store frequent queries, support MFU/LFU/LRU caching strategies, and pre-load data based on user behavior.
*   **Aggressive S3 Vectors indexing:** Each query hits just one index. A single S3 bucket can hold 10K indexes, each with ≤50M vectors. Smart indexing helps reduce latency significantly.

All of this requires solid engineering chops — but it's necessary if you want to build a cost-effective vector database with 10–100 ms latency on top of S3 Vectors.

Stay tuned for Level 2!

Learn how to build a cost-efficient vector database using Amazon S3 Vectors, Product Quantization, and open-source tools like JECQ and Redis — without sacrificing performance.

Build a Vector Database on Amazon S3 Vectors and Cut SaaS Costs

How to Build a Vector Database Using Amazon S3 Vectors

medium_benede.jpg

small_benede.jpg

thumbnail_benede.jpg

benede.jpg

Benedetto

Users are hesitant to store their data on a third-party platform due to security concerns and regulatory compliance.

Data Privacy & Security 

Integrating with a diverse and growing number of customer databases (SQL, NoSQL, SaaS platforms) required significant engineering effort.

Integration Complexity

The platform needed to handle complex, multi-record submissions with internal dependencies. 

Record Dependencies

Customer databases featured a wide array of primary key strategies (auto-incrementing, natural keys, UUIDs).  

Ambiguous Primary Key Handling

We designed the foundational architecture for the Data Fabric that orchestrates CRUD requests. 

Data Fabric Architecture 

Our team created a type-mapping system to handle a variety of data types and their "flavors" across different database systems.  

Dynamic Type Mapping 

The system effectively handles various primary key generation strategies (auto-generated, natural keys, UUIDs, custom sequences).  

Effective Key Inference 

We simplified the development & deployment of new integrations, so each new integration can be built by one developer in about a month. 

Rapid Third-Party Integration Flow

Janea Systems built Intellistack’s no-code platform, enabling secure CRUD operations and reducing time-to-market for new integrations.

Low-Code Solution for Secure CRUD Without Data Retention 

Architecting a Low-Code Solution for Secure CRUD Without Data Retention 

Choosing the right filesystem and architecture for historical tracking and ML applications.

Architecture Considerations

Helping the team allocate resources for the solution’s deployment and ongoing maintenance.

Workforce Limitations

A scalable data lake capable of processing multiple streams of historical data in parallel.

Robust Data Pipelines

A structured data lake facilitating long-term storage and efficient data retrieval.

Flexible Data Storage

A system designed for machine learning integration, enabling predictive modeling.

Built-in Support for Machine Learning

Horizontal scaling ensures future data growth while maintaining cost efficiency, fault tolerance, and optimal performance.

Scalability

Strict access controls, encrypted credential storage, and automated CI/CD pipelines guarantee data integrity and protection.

Enhanced Security

How to future-proof historical data for AI: scalable ETL pipelines, Delta Lake SCD2, Azure Synapse performance, and Key Vault security for a fintech platform [case study]

AI-Ready Data Lake For A Fintech Innovator

Borrower behavior is unpredictable, and the system must handle a wide variety of exceptions.

Edge cases

Many critical steps live in the day-to-day judgment of collectors.

SME expertise

LLM-based systems can vary in output while policies and process rules need reliable outcomes.

Non-determinism

Purpose-built agents supporting 15 to 50k cases a day, depending on the context.

Role-Specific Assistants

Shared orchestration and composable components for adding new assistants across channels without rebuilding the system.

Extensible Architecture

Improved completion of real-world workflows with tight feedback loops from SMEs and edge-case driven design.

Workflow Completion

A modular AI agent factory for finance that scales role-specific assistants across delinquency workflows, supporting between 15 to 50k cases a day.

AI Agent Factory for Finance

Oncologists spend 10-20 minutes per patient visit on manual note-taking instead of patient care. 

Operational Bottlenecks 

Lack of systematic coordination between doctors, nurses, and billing staff.

Fragmented Communication 

Manual medical coding caused claim rejections and required hiring scrubbing experts.  

Manual Medical Coding

Importing patient history from Integra while maintaining document accessibility for doctors.

Legacy Vendor Integration 

The existing RCM system couldn’t scale and needed a replacement.

Outdated Billing Infrastructure

Testing AI clinical notes required validation by oncologists.  

AI Quality Assurance 

50,000 tokens/minute restriction limited AI summary capabilities. 

Token Quota Limitations

Serving diverse user groups with different priorities and technical literacy levels.  

Cross-Functional Requirements 

Oncologists save 10-20 minutes per visit with an AI note generator.  

AI Clinical Summaries  

Converts doctor-patient conversations into structured progress notes. 

Ambient Listening 

AI-assisted generation of medical codes for insurance claims.  

AI Scribe for Medical Coding

Custom OCR processing for PDF documents for high-volume document management. 

Low-Cost OCR Processing 

Queue-based task management serving doctors, nurses, and billing teams.  

Healthcare Workflow Automation

Instantly answers any question, generates letters of medical necessity, and appeals.

AI for Prior Authorization 

An automated RCM system reduces claim rejections and scrubbing costs.  

Revenue Cycle Management 

Supports 40-50 administrative users across billing teams with capacity for growth. 

Scalable Infrastructure 

Our team created a clinical workflow management system for oncology practice with a built-in healthcare AI agent and clinical co-pilot. 

Healthcare AI Assistant with AI Medical Scribe 

Healthcare AI Assistant with AI Scribe, Ambient Listening, and RCM 

Head of Architecture

large_Janea Systems OG Image No Text.png

medium_Janea Systems OG Image No Text.png

small_Janea Systems OG Image No Text.png

thumbnail_Janea Systems OG Image No Text.png

Janea Systems OG Image No Text.png

Janea Systems is a software engineering consultancy, serving the enterprises that drive global innovation for over two decades. We help our clients build and optimize technologies used by millions of developers and billions of consumers. 

Projects

Intellistack

NDA

NDA

NDA

Related Blogs

Let's talk about your project

Let's talk about your project