Multitenant Vector Search in BigQuery ML: Why Tenant Pre-Filtering Tanks Recall

The tenant filter you wrote as a pre-filter stops being one the moment you attach a row access policy to the table. Google's own BigQuery documentation states two facts that, put side by side, decide how multitenant semantic search behaves: stored columns, the mechanism that lets a WHERE tenant_id = ... clause run before the approximate nearest neighbor (ANN) scan, "are not used if the table has a row-level access policy," and when a filter cannot use stored columns it acts as a post-filter, so "the final result set might contain fewer than top_k rows, potentially even zero rows, if the predicate is selective." A tenant predicate is about as selective as predicates get. So the standard way to secure a shared embeddings table in BigQuery ML, row-level security plus VECTOR_SEARCH, routes every tenant's query through search-then-filter. Search-then-filter is the configuration that the published filtered-ANN literature shows collapsing at low selectivity: in Qdrant's August 2026 benchmark, a plain HNSW graph searched with a 1% selectivity filter recovered 0.1% of the true neighbors, and in AWS's pgvector 0.7.4 baseline, a category filter returned 10% of the expected results. Google does not publish a recall figure for this exact BigQuery scenario, and I will not invent one. What follows is the documented mechanism, the real numbers from other systems that share it, the SQL to measure it on your own table, and the fixes that the documentation actually supports.

# What this article covers

1. Why This Matters: The Filter You Wrote Is Not the Filter That Runs
2. Concepts You Need Before We Get to the Numbers
3. How BigQuery ML Actually Executes a Tenant-Filtered VECTOR_SEARCH
4. The Evidence: What Published Benchmarks Say About Filtered ANN Recall
5. The Fix, End to End: Building Halyard's Tenant-Safe Search in BigQuery ML
6. Pitfalls and Troubleshooting
7. What Halyard Shipped, and What the Finding Means for Your Tenant Model

# Why This Matters: The Filter You Wrote Is Not the Filter That Runs

Multitenant search has a security requirement and a relevance requirement, and in most stacks they pull against each other quietly. The security requirement says tenant A must never see tenant B's documents. The relevance requirement says tenant A's query must return tenant A's ten best documents, not three of them, and not zero.

In BigQuery ML the obvious design satisfies the security requirement and quietly breaks the relevance requirement. You put every tenant's chunks and embeddings in one table, build one vector index, attach a row access policy keyed on tenant_id, and call VECTOR_SEARCH. Every piece is documented and works. The trouble is in how the pieces compose, which you only see by reading the vector index guide, the VECTOR_SEARCH reference, and the row-level security pages together.

# The case we will follow: Halyard Desk (fictional)

Who: Halyard Desk is a fictional B2B helpdesk SaaS. Each customer (a tenant) uploads its own help center articles, macros, and resolved ticket summaries, and Halyard's "suggested answers" panel uses semantic search to surface the most relevant internal content while a support agent is typing a reply.
The situation: Halyard stores every tenant's content chunks and their embeddings in a single BigQuery table, generates embeddings with BigQuery ML's ML.GENERATE_EMBEDDING, and serves suggestions with VECTOR_SEARCH over an IVF vector index. To satisfy enterprise security reviews, the data team added a row access policy so that each tenant's service identity can only read its own rows.
What broke: After the row access policy shipped, agents at smaller tenants started reporting that the suggested answers panel showed two or three suggestions instead of ten, and sometimes none at all, even for questions their own help center clearly answered. Large tenants saw no change.
What it cost: A run of support escalations from the tenants least able to absorb a broken feature, an engineering investigation that initially chased the embedding model, and a security review that the team did not want to reopen.
Where we end up: A tiered design: exact brute-force search for small tenants, over-fetch with a re-ranking trim for the middle, and physical isolation for the largest tenants, plus a recall harness in SQL that runs on every index change. No part of it depends on an undocumented BigQuery behavior.

Nothing in Halyard's stack is misconfigured. The recall loss is structural, the same loss that vector database vendors and researchers have measured for years under the name "filtered ANN search."

# Concepts You Need Before We Get to the Numbers

Search-then-filter is a mouthful of jargon until you have the individual pieces straight. They need to be separate for the rest of this piece to make sense.

The two execution paths that matter look like this.

Stored columns make the tenant filter a true pre-filter on the left path. A row access policy on the table removes that path entirely, even for an administrator's query.

# How BigQuery ML Actually Executes a Tenant-Filtered VECTOR_SEARCH

Every link in this chain is a quote from Google Cloud documentation ("Manage vector indexes," the VECTOR_SEARCH reference, and the row-level security pages), so you can check each one yourself.

# Link 1: pre-filtering requires stored columns

The vector index guide says that to create a pre-filter, "your WHERE clause must apply to the base table within the search function and reference stored columns." The VECTOR_SEARCH reference repeats it from the other side: "If the base table is indexed and the WHERE clause contains columns that are not stored in the index, then VECTOR_SEARCH post-filters on those columns instead."

So STORING (tenant_id) on the index plus WHERE tenant_id = 'acme' inside the base table subquery is a genuine pre-filter. The index searches only Acme's rows.

# Link 2: row access policies switch stored columns off

The same vector index guide contains this sentence, in the stored columns section:

"Stored columns are not used if the table has a row-level access policy or the column has a policy tag."

Note the scope: it is the table having a policy that disables stored columns, not the querying user being subject to one. Once Halyard attached its policy, STORING (tenant_id) became dead weight for every query on that table, including an administrator's.

# Link 3: the tenant filter becomes a post-filter, and post-filters underfill

With stored columns unavailable, the tenant WHERE clause falls under Link 1's second sentence, and VECTOR_SEARCH post-filters on it. The vector index guide is explicit about the consequence:

"When you use post-filtering, or when the base table filters you specify reference non-stored columns and thus act as post-filters, the final result set might contain fewer than top_k rows, potentially even zero rows, if the predicate is selective."

# Link 4: the policy itself is applied to results

Separately from the explicit WHERE clause, the VECTOR_SEARCH limitations list says: "If the base table has row-level security policies, VECTOR_SEARCH applies the row-level access policies to the query results."

Read literally, that also describes filter-after-search semantics for the policy predicate. Google does not document whether, in brute-force mode, the policy predicate is pushed below the top-k selection or evaluated after it. I flag this as unknown rather than guess, and the recall harness later in this article is designed to tell you which behavior your table exhibits.

# What Google does and does not publish

That last row is the honest boundary of this article. There is no Google-published recall@k number for VECTOR_SEARCH under a tenant filter, so every hard recall figure below comes from other systems, and I say which.

# Why the mechanism guarantees a loss, without needing a BigQuery number

You do not need a benchmark to see the shape of the problem; you need arithmetic. Suppose the index returns a candidate list of size k from the whole table and the tenant filter is applied afterward. If the tenant's rows were scattered independently of the query, the expected number of survivors would be the tenant's selectivity multiplied by k. The pgvector project documents exactly this calculation for its own post-filtered indexes: "If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, only 4 rows will match on average." Different system, same arithmetic, and the README is a primary source.

Real tenants are not scattered independently of their own queries. A tenant's queries tend to land near its own content, which helps. Qdrant's benchmark (discussed below) shows how much: on a 10% selectivity filter, the uncorrelated case kept 20.6% recall on a plain graph while the correlated case kept 88.4%. Halyard's problem is that helpdesk tenants are not well separated. Every software company's help center has an article about resetting a password, configuring single sign-on, and exporting an invoice. For a query like "user cannot log in after SSO change," the true global nearest neighbors are the near-duplicate SSO articles of many other tenants. They crowd the candidate list, the post-filter deletes them, and the small tenant's own SSO article never made it into the list to begin with. Homogeneous tenants are the worst case for search-then-filter, and B2B SaaS tenants are usually homogeneous.

# The Evidence: What Published Benchmarks Say About Filtered ANN Recall

Every number in this section comes from a named, public source, and none of it was measured on BigQuery. I include the system, dataset, and hardware for each so you can judge how far it transfers. After the tables I explain which of them map onto BigQuery's IVF post-filter path and which do not.

# Qdrant: filtered HNSW recall at controlled selectivities

Qdrant's engineering blog post "Filtered Vector Search: ACORN and Filterable HNSW" (Dylan Couzon and Meina Ghafouri, August 7, 2026) is the cleanest published sweep I found. Couzon and Ghafouri state that "every number in this article was measured on Qdrant v1.18.2, on one laptop-class machine, queried serially," on one million deep-image-96 vectors, with recall scored "against exact brute force over 500 queries per filter." The table below reproduces their single-filter results at hnsw_ef=64, each cell as recall at latency.

Source: Qdrant blog, measured on Qdrant v1.18.2, laptop-class machine, deep-image-96 (1M vectors).

Two lessons matter for Halyard. First, the plain graph's recall falls from 62.9% to 0.1% as selectivity drops from 20% to 1%, the range where many small tenants of a shared table live. Second, correlation matters enormously: at the same 10% selectivity, the correlated filter keeps 88.4% where the uncorrelated filter keeps 20.6%. The post explains the graph failure mechanically: "Filter out 96% of the points and fewer than one link per node survives on average, so traversal can get stranded before it reaches the true nearest matches."

The same post also reports Qdrant's default query planner, which can choose not to traverse the graph for very selective filters. With ACORN off, the planner returned 100% recall at 1.4 ms for a two-keyword filter at 0.012% selectivity, measured on the same Qdrant v1.18.2 setup. Skipping the approximate structure for tiny filtered sets is the strategy behind BigQuery's use_brute_force tier for Halyard's small tenants later on.

# AWS and pgvector: post-filtering returns a fraction of the rows you asked for

The AWS Database Blog post "Supercharging vector search performance and relevance with pgvector 0.8.0 on Amazon Aurora PostgreSQL" (Shayon Sanyal, May 28, 2025) benchmarked 10 million products with 384-dimensional all-MiniLM-L6-v2 embeddings on an Aurora db.r8g.4xlarge (AWS Graviton4) instance. Its definition of recall is about completeness: "Recall means we return X out of Y expected results, with 100% being perfect recall." pgvector 0.7.4 applied the filter after the index scan, which the post calls "overfiltering"; pgvector 0.8.0 added iterative index scans that keep searching until enough rows pass the filter.

Source: AWS Database Blog, benchmarked on Aurora PostgreSQL db.r8g.4xlarge, 10M rows, 384 dimensions.

This is the closest published analog to Halyard's symptom, because it is measured as underfill: the query asked for 10 rows and got back a tenth of them. The post states the cause directly: "filtering happened after the vector index scan completed," so queries could "return fewer results than expected, or even none at all." That is nearly word for word what BigQuery's documentation says about post-filters. (The post also reports 0% recall for the 0.7.4 runs at ef_search=200 on the two filtered queries; I report it as published, but I leave it out of the table because the post does not explain why the larger search parameter did worse.)

# NeurIPS 2023 Big-ANN: the IVF-then-filter baseline

The "Results of the Big ANN: NeurIPS'23 competition" paper (Simhadri, Aumüller, Ingber, Douze, and coauthors) included a filtered track on 10 million YFCC images encoded as 192-dimensional CLIP embeddings, each with a bag of tags, ranked by throughput at or above 90% 10-recall@10. The organizers' baseline was built on Faiss and had two modes that map almost exactly onto BigQuery's two options. In the paper's words: "In vector-first mode, the search is performed with a Faiss IVF index and vector results that do not satisfy the word constraint are removed from the result list. In metadata-first mode, the database is reduced to the vectors satisfying the word constraint; in that case the vector search is performed in brute force."

Source: NeurIPS'23 Big-ANN results paper, measured on an Azure D8lds_v5 virtual machine (8 vCPUs, 16 GB RAM), YFCC-10M.

This is a throughput table: entries were ranked by QPS at the 90% recall bar. Its value for us is structural. The organizers' baseline combined an IVF post-filter path with a brute-force pre-filter path, which is mechanically what BigQuery offers through the index and use_brute_force, and filter-aware designs beat it by roughly an order of magnitude at the same recall.

# ACORN and Filtered-DiskANN: what filter-aware indexes buy

Two research systems show what is possible when the index is built with filters in mind. I cite them for context, not as BigQuery numbers, and BigQuery ML does not expose either algorithm.

• ACORN (Patel, Kraft, Guestrin, Zaharia, SIGMOD 2024) extends HNSW with predicate subgraph traversal. The paper reports between 2 and 1,000 times higher throughput than prior methods at a fixed recall, including "over 1,000x higher QPS at scale, on a 25-million-vector dataset," with experiments on an AWS m5d.24xlarge instance with 96 vCPUs and 370 GB of RAM across SIFT1M, Paper, TripClick, and LAION datasets. The paper also explains why the post-filter baseline must over-search: for selectivity s, it gathers "K/s candidate results before applying the query filter," which is expensive exactly when s is small.
• Filtered-DiskANN (Gollapudi and colleagues, Microsoft Research, The ACM Web Conference 2023) builds graph edges using both vector geometry and label sets. Its abstract reports that both of its algorithms are "an order of magnitude or more efficient for filtered queries than the current state of the art," and that the indexes "support thousands of queries per second at over 90% recall@10" when served from SSD.

# What transfers to BigQuery, and what does not

The defensible claim is therefore narrower and stronger than "BigQuery has bad recall." It is: BigQuery documents that a row access policy forces tenant filters onto the post-filter path, and every published measurement of the post-filter path at low selectivity, on every system I could find, shows severe underfill. How severe on your table depends on tenant size, how similar tenants' content is, top_k, and fraction_lists_to_search. The next section gives you the SQL to find out.

# The Fix, End to End: Building Halyard's Tenant-Safe Search in BigQuery ML

The walkthrough below is the design Halyard ends up with. All statements use syntax from the current BigQuery documentation. Project, dataset, group, and service account names are fictional; replace them with your own. I use # for SQL comments throughout, which GoogleSQL supports.

# Step 1: the table and the embedding model

Halyard stores one row per content chunk. The embedding comes from a BigQuery ML remote model over a Vertex AI embedding endpoint. The ML.GENERATE_EMBEDDING reference lists text-embedding-005 and gemini-embedding-001 among the supported text endpoints; use whichever your region and quota allow.

# Remote embedding model, served through a Cloud resource connection.
CREATE OR REPLACE MODEL `halyard-prod.kb.embedder`
  REMOTE WITH CONNECTION `halyard-prod.us.vertex_conn`
  OPTIONS (ENDPOINT = 'text-embedding-005');

# One row per chunk, all tenants in one table.
CREATE TABLE IF NOT EXISTS `halyard-prod.kb.chunks` (
  chunk_id   STRING NOT NULL,
  tenant_id  STRING NOT NULL,
  doc_type   STRING,
  title      STRING,
  content    STRING,
  embedding  ARRAY<FLOAT64>
);

ML.GENERATE_EMBEDDING sends the column named content to the model and returns the vector in ml_generate_embedding_result. Halyard backfills embedding from a staging table with task_type set to 'RETRIEVAL_DOCUMENT', and filters on ml_generate_embedding_status to drop rows whose embedding call failed. Queries use 'RETRIEVAL_QUERY', as shown below.

# Step 2: the vector index, with the tenant column stored

CREATE OR REPLACE VECTOR INDEX chunks_ivf
ON `halyard-prod.kb.chunks`(embedding)
STORING (tenant_id, doc_type)
OPTIONS (
  index_type = 'IVF',
  distance_type = 'COSINE'
);

I leave ivf_options out so BigQuery picks num_lists from the data. If you set it, the documented ceiling is 5,000, and the docs note that more lists let you scan "a smaller percentage of the index" with fraction_lists_to_search.

Two operational details from the vector index guide matter here. Indexing is asynchronous, but VECTOR_SEARCH still "take[s] all rows into account and [doesn't] miss unindexed rows." And "if you create a vector index on a table that is smaller than 10 MB, then the vector index isn't populated," so a small development table silently runs brute force. Check index health before measuring anything:

SELECT
  index_name,
  index_status,
  coverage_percentage,
  last_refresh_time
FROM `halyard-prod.kb.INFORMATION_SCHEMA.VECTOR_INDEXES`
WHERE table_name = 'chunks';

At this point, with no row access policy, a tenant query is a true pre-filter, because tenant_id is stored:

SELECT query.query_text, base.chunk_id, base.title, distance
FROM VECTOR_SEARCH(
  (SELECT chunk_id, tenant_id, title, embedding
   FROM `halyard-prod.kb.chunks`
   WHERE tenant_id = 'tnt_harbor_kayaks'),
  'embedding',
  (SELECT content AS query_text, ml_generate_embedding_result AS embedding
   FROM ML.GENERATE_EMBEDDING(
     MODEL `halyard-prod.kb.embedder`,
     (SELECT 'customer cannot log in after SSO change' AS content),
     STRUCT(TRUE AS flatten_json_output, 'RETRIEVAL_QUERY' AS task_type))),
  top_k => 10,
  distance_type => 'COSINE'
);

Note the base table subquery uses only SELECT, FROM, and WHERE, which is all the reference allows, and it does not filter the embedding column.

# Step 3: the row access policy that changes everything

Halyard's security review asked for enforcement in the data layer, so the team built the standard lookup-table policy from the row-level security docs. Each tenant's backend calls BigQuery as its own service account, and the lookup table maps that identity to a tenant.

CREATE TABLE IF NOT EXISTS `halyard-prod.kb.tenant_principals` (
  principal_email STRING NOT NULL,
  tenant_id       STRING NOT NULL
);

CREATE OR REPLACE ROW ACCESS POLICY tenant_isolation
ON `halyard-prod.kb.chunks`
GRANT TO ('group:[email protected]')
FILTER USING (
  tenant_id IN (
    SELECT tenant_id
    FROM `halyard-prod.kb.tenant_principals`
    WHERE principal_email = SESSION_USER()
  )
);

Halyard also needs one identity that can read every row, both for maintenance and for measuring recall against an exact ground truth:

CREATE OR REPLACE ROW ACCESS POLICY recall_eval_full_read
ON `halyard-prod.kb.chunks`
GRANT TO ('serviceAccount:[email protected]')
FILTER USING (TRUE);

Querying under a policy requires bigquery.tables.getData plus bigquery.rowAccessPolicies.getFilteredData, and the docs warn that every principal in a GRANT TO list must already exist or the statement fails.

From this moment, per the vector index guide, STORING (tenant_id, doc_type) is not used for any query on chunks. The Step 2 query still runs and still returns only Harbor Kayaks' rows, so nothing looks broken. But its WHERE tenant_id = ... is now a post-filter, and the policy is applied to the results on top of it.

# Step 4: confirm the index is in use, and count the underfill

Before measuring recall, confirm what BigQuery did. The INFORMATION_SCHEMA.JOBS view has a vector_search_statistics column, documented as "statistics for a vector search query," whose index usage mode is FULLY_USED, PARTIALLY_USED, or UNUSED, with a list of reasons when the index was not fully used.

SELECT
  job_id,
  creation_time,
  vector_search_statistics.index_usage_mode AS index_usage_mode,
  vector_search_statistics.index_unused_reasons AS index_unused_reasons,
  total_slot_ms
FROM `halyard-prod.region-us.INFORMATION_SCHEMA.JOBS`
WHERE creation_time > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR)
  AND vector_search_statistics IS NOT NULL
ORDER BY creation_time DESC;

FULLY_USED tells you the index path ran; it does not tell you how many rows survived the filter. The harness in Step 5 counts that too, because underfill is the symptom Halyard's agents saw, and a query returning fewer than 10 rows for a tenant that owns more than 10 chunks is direct evidence of post-filter underfill with no assumption about internals.

# Step 5: measure recall against brute force

Underfill catches missing rows; recall catches wrong rows too. Run this as the recall-eval service account, whose policy grants every row, so that the explicit WHERE is the only tenant restriction. Because the table has a row access policy, that WHERE is a post-filter on the index path, which reproduces production. The brute-force path gives exact ground truth for the tenant.

DECLARE target_tenant STRING DEFAULT 'tnt_harbor_kayaks';

CREATE TEMP TABLE eval_q AS
SELECT query_id, embedding
FROM `halyard-prod.kb.eval_queries`
WHERE tenant_id = target_tenant;

WITH ann AS (
  SELECT query.query_id AS query_id, base.chunk_id AS chunk_id
  FROM VECTOR_SEARCH(
    (SELECT chunk_id, tenant_id, embedding
     FROM `halyard-prod.kb.chunks`
     WHERE tenant_id = target_tenant),
    'embedding',
    (SELECT query_id, embedding FROM eval_q),
    top_k => 10,
    distance_type => 'COSINE'
  )
),
exact AS (
  SELECT query.query_id AS query_id, base.chunk_id AS chunk_id
  FROM VECTOR_SEARCH(
    (SELECT chunk_id, tenant_id, embedding
     FROM `halyard-prod.kb.chunks`
     WHERE tenant_id = target_tenant),
    'embedding',
    (SELECT query_id, embedding FROM eval_q),
    top_k => 10,
    distance_type => 'COSINE',
    options => '{"use_brute_force": true}'
  )
),
ann_counts AS (
  SELECT query_id, COUNT(*) AS ann_returned
  FROM ann
  GROUP BY query_id
),
per_query AS (
  SELECT
    e.query_id,
    COUNT(DISTINCT e.chunk_id) AS exact_rows,
    COUNT(DISTINCT a.chunk_id) AS ann_rows_in_exact
  FROM exact AS e
  LEFT JOIN ann AS a
    ON a.query_id = e.query_id AND a.chunk_id = e.chunk_id
  GROUP BY e.query_id
)
SELECT
  target_tenant AS tenant_id,
  COUNT(*) AS queries,
  AVG(SAFE_DIVIDE(p.ann_rows_in_exact, p.exact_rows)) AS mean_recall_at_10,
  AVG(COALESCE(c.ann_returned, 0)) AS mean_rows_returned,
  COUNTIF(COALESCE(c.ann_returned, 0) < p.exact_rows) AS underfilled_queries,
  COUNTIF(COALESCE(c.ann_returned, 0) = 0) AS empty_queries
FROM per_query AS p
LEFT JOIN ann_counts AS c USING (query_id);

Three design choices in this harness are deliberate:

1. Ground truth uses the same tenant WHERE, and underfill is counted from the ANN side. If a tenant owns fewer than ten chunks, exact search returns fewer than ten rows, so recall divides by exact_rows rather than by 10. The LEFT JOIN to ann_counts keeps queries whose ANN search returned nothing, which a plain GROUP BY over the ANN output would silently drop.
2. Results are reported per tenant. A table-wide average is dominated by the largest tenants, the ones that suffer least. Halyard runs this over a sample of tenants bucketed by row count and plots recall against selectivity, reproducing the shape of the published curves on its own data.
3. One extra run answers the open question from earlier. Run the exact CTE once more as a tenant's own service identity, with no explicit WHERE, so the row access policy is the only tenant restriction. If that brute-force run returns fewer rows than the eval account's run, the policy predicate is being applied after top-k selection on your table, and brute force is not a safe exact path under the policy. You learn this empirically rather than assuming it.

Halyard's own harness numbers are not reproduced here, because Halyard is fictional and I will not dress up made-up numbers as measurements.

# Step 6: mitigation A, over-fetch and trim

The cheapest fix keeps the row access policy and attacks the arithmetic: if survivors scale with the candidate list length, make the list longer. Raise top_k well beyond what the UI (the BigQuery console's results pane) shows, raise fraction_lists_to_search so the IVF scan visits more lists, then keep the ten closest survivors.

SELECT query_text, chunk_id, title, distance
FROM (
  SELECT
    query.query_text AS query_text,
    base.chunk_id AS chunk_id,
    base.title AS title,
    distance,
    ROW_NUMBER() OVER (PARTITION BY query.query_text ORDER BY distance) AS rn
  FROM VECTOR_SEARCH(
    (SELECT chunk_id, tenant_id, title, embedding
     FROM `halyard-prod.kb.chunks`
     WHERE tenant_id = @tenant_id),
    'embedding',
    (SELECT content AS query_text, ml_generate_embedding_result AS embedding
     FROM ML.GENERATE_EMBEDDING(
       MODEL `halyard-prod.kb.embedder`,
       (SELECT @query_text AS content),
       STRUCT(TRUE AS flatten_json_output, 'RETRIEVAL_QUERY' AS task_type))),
    top_k => @overfetch_k,
    distance_type => 'COSINE',
    options => '{"fraction_lists_to_search": 0.2}'
  )
)
WHERE rn <= 10
ORDER BY distance;

The 0.2 is a starting point to tune, not a measured recommendation, and @overfetch_k is a query parameter so you can set it per tenant. The ACORN paper's post-filter baseline tells you how to reason about it: to expect K survivors at selectivity s you need on the order of K/s candidates. For a tenant with a tiny share of the table, K/s grows past anything reasonable, so over-fetching fixes the middle of the tenant distribution, not the tail. Pick values per size bucket with the Step 5 harness, and watch total_slot_ms as you raise them, since the docs promise "slower performance" in exchange for recall.

# Step 7: mitigation B, exact search for small tenants

For the long tail, give up on the index. A small tenant's rows are few by definition, so exact search is the right tool: the strategy Qdrant's planner takes at very low selectivity and that the Big-ANN Faiss baseline calls metadata-first mode.

SELECT query.query_text, base.chunk_id, base.title, distance
FROM VECTOR_SEARCH(
  (SELECT chunk_id, tenant_id, title, embedding
   FROM `halyard-prod.kb.chunks`
   WHERE tenant_id = @tenant_id),
  'embedding',
  (SELECT content AS query_text, ml_generate_embedding_result AS embedding
   FROM ML.GENERATE_EMBEDDING(
     MODEL `halyard-prod.kb.embedder`,
     (SELECT @query_text AS content),
     STRUCT(TRUE AS flatten_json_output, 'RETRIEVAL_QUERY' AS task_type))),
  top_k => 10,
  distance_type => 'COSINE',
  options => '{"use_brute_force": true}'
);

Two caveats. Check bytes processed in INFORMATION_SCHEMA.JOBS, since a scan of a large shared table is not priced by the tenant's size. And confirm with the Step 5 harness that brute force fills to top_k under the row access policy, since Google does not document where the policy predicate sits in that path. Halyard routes tenants between Steps 6 and 7 using a per-tenant row count table (COUNT(*) grouped by tenant_id), with cutoffs taken from the harness output rather than a rule of thumb.

# Step 8: mitigation C, physical isolation for the largest tenants

Large tenants have the highest selectivity and so suffer least under post-filtering, but they still pay for scanning everyone else's lists. The cleanest fix is to stop sharing: give each large tenant its own table in its own dataset, secured with dataset-level IAM, which is Google Cloud's Identity and Access Management system and grants access per dataset rather than per row, instead of a row access policy. With no policy, there is nothing to disable and no tenant predicate to filter.

CREATE SCHEMA IF NOT EXISTS `halyard-prod.kb_tnt_northwind`;

CREATE OR REPLACE TABLE `halyard-prod.kb_tnt_northwind.chunks` AS
SELECT chunk_id, doc_type, title, content, embedding
FROM `halyard-prod.kb.chunks`
WHERE tenant_id = 'tnt_northwind';

CREATE OR REPLACE VECTOR INDEX chunks_ivf
ON `halyard-prod.kb_tnt_northwind.chunks`(embedding)
STORING (doc_type)
OPTIONS (index_type = 'IVF', distance_type = 'COSINE');

On this table, STORING (doc_type) works again, so a secondary filter such as "only help center articles" is a true pre-filter. Remember the 10 MB floor: a tenant table below it has no populated index and runs brute force, which is fine for correctness but means it belongs in the Step 7 tier rather than here.

# Step 9: an option to test, not to assume: a partitioned TreeAH index

BigQuery supports partitioned vector indexes, but only for TreeAH: "You can only partition TreeAH vector indexes." The supported partitioning types include integer range partitioning, and the index's PARTITION BY "must be the same as the PARTITION BY clause specified in the CREATE TABLE statement." If each tenant has an integer tenant number, a tenant-partitioned table and index look like this:

CREATE TABLE `halyard-prod.kb.chunks_by_tenant` (
  chunk_id    STRING,
  tenant_num  INT64,
  title       STRING,
  embedding   ARRAY<FLOAT64>
)
PARTITION BY RANGE_BUCKET(tenant_num, GENERATE_ARRAY(0, 4000, 1));

CREATE VECTOR INDEX chunks_tree_ah
ON `halyard-prod.kb.chunks_by_tenant`(embedding)
PARTITION BY RANGE_BUCKET(tenant_num, GENERATE_ARRAY(0, 4000, 1))
OPTIONS (index_type = 'TREE_AH', distance_type = 'COSINE');

To use the partitioned index, the docs say to "filter on the partitioning column in the base table subquery of the VECTOR_SEARCH or AI.SEARCH call." Partition pruning is a different mechanism from stored columns, so in principle it could confine the search to one tenant's partition even under a row access policy. I treat it as an experiment for three reasons: row-level security "does not participate in query pruning," so the explicit partition filter is mandatory; Google's TreeAH guidance targets large query batches, not per-agent search; and I found no Google statement on whether vector index partition pruning stays active on a table with a row access policy. Build it in a scratch dataset, run the Step 4 and Step 5 queries against it, and check the quotas page for the per-table partition limit before choosing your range.

# The alternative Halyard rejected: drop the policy, filter in the app

One configuration restores true pre-filtering on the shared table: remove the row access policy, keep STORING (tenant_id), restrict the table to one backend service account, and inject WHERE tenant_id = @tenant_id on every call. The cost is that tenant isolation moves into application code, exactly what Halyard's security review wanted to avoid, so Halyard kept the policy and worked around its effect on recall.

# Pitfalls and Troubleshooting

Each of these failure modes ties back to a documented behavior.

1. Your dev table is too small to have an index: Under 10 MB, the vector index "isn't populated" and queries fall back to brute force. Every filtered query in development returns perfect, full results, and the problem appears only in production. Always check INFORMATION_SCHEMA.VECTOR_INDEXES and the job's index_usage_mode before trusting a test.

2. You measured recall during an index refresh: Indexing is asynchronous, so a low coverage_percentage means part of the table is searched differently from the rest and your numbers describe a transient state. Measure once coverage settles.

3. A policy tag on tenant_id has the same effect as a row access policy: The disabling sentence covers both: stored columns are not used "if the table has a row-level access policy or the column has a policy tag." Teams that move from row-level security to column-level classification sometimes reintroduce the problem without noticing.

4. The tenant filter lives in a subquery the index cannot see: The VECTOR_SEARCH reference allows only SELECT, FROM, and WHERE in the base table query and warns that subqueries might interfere with index usage. Keep the tenant predicate a plain comparison on a column, and resolve the tenant ID in the application or a query parameter rather than with a join inside the base query.

5. fraction_lists_to_search and use_brute_force in the same options string: The reference states you "can't specify fraction_lists_to_search when use_brute_force is set to true." Routing code that merges option dictionaries can produce exactly this combination.

6. The row access policy does not prune partitions: If you go the partitioned route in Step 9, the policy alone will not restrict the scan. Add the explicit partition-column filter in the base table subquery.

7. Lookup-table policies have a size ceiling: The row-level security overview lists a 100 MB limit on the results of subqueries used in policies. Map service identities to tenants rather than every human user, so tenant_principals stays small.

8. Averages hide the tail: A single recall number for the whole table will look healthy because large tenants dominate it. The tenants that complain are the small ones. Always report recall by tenant size bucket.

9. Timing is a side channel: The row-level security overview warns that "side channels, such as the query duration, can leak information about rows." Tiered routing makes latency depend on tenant size; disclose that in the security review.

10. Cross-project splits break queries: "The project that runs the query containing VECTOR_SEARCH must match the project that contains the base table." Isolate large tenants by dataset, not by project, unless you also move the job project.

# What Halyard Shipped, and What the Finding Means for Your Tenant Model

Halyard's final design has three tiers chosen per tenant from the harness output: use_brute_force for the long tail, over-fetch plus a ROW_NUMBER trim for the middle, and dedicated tables with their own IVF index for the largest tenants. The row access policy stays on the shared table, so the security review stays closed, and the Step 5 harness runs whenever the index or embedding model changes, reported per size bucket.

The three routing tiers Halyard shipped, chosen per tenant from the Step 5 recall harness rather than a fixed rule of thumb.

The general finding is about how documented features compose. BigQuery ML gives you a correct pre-filter (stored columns), a correct isolation mechanism (row access policies), and an honest statement that post-filters underfill. Put the first two on the same table and the first one switches off, which turns every tenant query into a search-then-filter query. The literature has measured search-then-filter at low selectivity many times on many systems, benchmarked on their own hardware, none of it on BigQuery:

None of those are BigQuery numbers, and Google has not published BigQuery's. They do not need to be BigQuery numbers for the conclusion to hold, because the mechanism they measure is the one BigQuery's documentation describes.

So the question to ask of any multitenant vector design, in BigQuery or anywhere else, is not "is the tenant filter correct?" It is "where does the tenant filter run relative to the index?" In BigQuery ML, the answer changes the moment a row access policy appears on the table, and the only way to know what it costs you is to measure recall per tenant against brute force on your own data.

If you want to contact me, feel free to drop an e-mail at [email protected] or check out my website at adityaseth.in :)
Also, here's my LinkedIn.

Thank you everyone for reading,

Over and out,
Aditya Seth.