au-property-embed-v2

A text embedding model for searching Australian property text: sale and rental listings, agent profiles and suburb descriptions. It is Alibaba-NLP/gte-modernbert-base (149M parameters) finetuned on Australian property search queries, with general question-answer pairs mixed in so it keeps the base model's general retrieval ability.

Status: research prototype. It shows that the method works on synthetic data. It has not been tested on real listings and is not intended for production use.

  • Property search: nDCG@10 rises by 0.23 to 0.30 over the base model and 0.21 to 0.23 over BM25 on four test suites frozen before training.
  • General retrieval kept: mean nDCG@10 on five NanoBEIR sets is 0.751, against 0.755 for the base model.
  • Smaller embeddings: trained with a Matryoshka loss, so embeddings can be cut from 768 to 512 dimensions (about 0.01 nDCG@10 lost) or 256 (0.02 to 0.03 lost). int8 embeddings lose nothing.

Independence and provenance

This is a personal research project, built in my own time on my personal laptop, phone and Claude account. No employer resources (hardware, accounts, compute, data or tools) were used at any stage. It is not affiliated with, sponsored by, or endorsed by any employer, and it does not represent the views of any organisation I work for.

  • No proprietary data. No commercial, employer or portal data was used: no listings, search logs, user data or internal datasets, and no scraped portal content.
  • Open data only. Every external input is openly licensed and listed under Training data.
  • Synthetic listings. Listing text, agent profiles and queries were generated by an LLM from records sampled from open data. They do not describe any real property or agent, and they are not copied from any portal.
  • AI-assisted. The code, data pipeline, training and evaluation were produced with an AI coding assistant (Claude) working with the author, and have not been independently reviewed.

Usage

Requires sentence-transformers 6 or later.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("dylanhogg/au-property-embed-v2")
# For smaller vectors: SentenceTransformer("dylanhogg/au-property-embed-v2", truncate_dim=256)

listings = [
    "12 Fig Street, Paddington QLD 4064\nClassic Queenslander with wraparound verandah, 3 bed, 2 bath.",
    "5/8 Beach Road, Bondi NSW 2026\nRenovated 2 bed unit, LUG, BIR, internal laundry.",
]
query = "character home near Brisbane with a verandah"

scores = model.similarity(model.encode([query]), model.encode(listings))
print(scores)
  • Similarity: cosine.
  • No prefixes: queries and documents are encoded the same way, with no instruction or prefix.
  • Document format: documents were embedded as the title (usually the address), a newline, then the body. Use the same format for best results.
  • Length: trained on inputs truncated to 256 tokens. max_seq_length is set to 512, and all evaluation used 512.
  • Pooling: CLS token, as in the base model.
  • GPU: the safetensors weights run on NVIDIA GPUs and Apple silicon with no separate model. sentence-transformers uses CUDA or MPS automatically when available. Pass device= to choose one yourself. v2 was trained and evaluated on an Apple M1 Max GPU (MPS), embedding about 160 documents a second.
  • Precision: the weights are stored in half precision, which suits GPUs. On a CPU, loading with model_kwargs={"dtype": torch.float32} is usually faster.

ONNX (int8, CPU)

onnx/model_int8.onnx (150 MB) is an int8-quantised export for CPUs without PyTorch. It takes input_ids and attention_mask and returns sentence_embedding, with pooling included. Normalise the output before taking dot products.

import numpy as np
import onnxruntime
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

repo = "dylanhogg/au-property-embed-v2"
session = onnxruntime.InferenceSession(hf_hub_download(repo, "onnx/model_int8.onnx"))
tokenizer = Tokenizer.from_file(hf_hub_download(repo, "tokenizer.json"))
tokenizer.enable_truncation(512)
tokenizer.enable_padding(pad_id=tokenizer.token_to_id("[PAD]"), pad_token="[PAD]")

encoded = tokenizer.encode_batch(["3 bedroom house for rent in Croydon VIC"])
embeddings = session.run(None, {
    "input_ids": np.array([e.ids for e in encoded], dtype=np.int64),
    "attention_mask": np.array([e.attention_mask for e in encoded], dtype=np.int64),
})[0]
embeddings /= np.linalg.norm(embeddings, axis=1, keepdims=True)

Quantising the weights costs quality: validation nDCG@10 falls from 0.722 to 0.641, and its embeddings have a mean cosine of 0.80 with the full model's. Prefer the safetensors weights, and use the ONNX file only where PyTorch isn't available and the loss is acceptable. Don't mix embeddings from the two in one index.

Intended use

  • Suitable for: research and prototyping of semantic search over Australian property text: natural-language queries such as "3br Croydon VIC rent max $425 pw" or "character home near Brisbane with a verandah" against listings, agent profiles and suburb descriptions.
  • Use alongside structured filters. Negation ("no pool"), budgets, and location that isn't stated in the text ("near Dubbo", "postcode 2460") are weak points. A real system should filter on price, location and feature fields and use the embedding for ranking.
  • Not suitable for: production use without evaluation on real data, valuation or any decision about people, or as a source of facts about suburbs or properties.

Results

nDCG@10 on test suites frozen before training. Higher is better.

System In-distribution (500 q) Gold (200 q) Unseen suburbs (300 q) Unseen writer (350 q) General
BM25 0.484 0.507 0.508 0.557 0.568
bge-base-en-v1.5 0.416 0.473 0.408 0.541
granite-embedding-english-r2 0.415 0.425 0.409 0.538
gte-modernbert-base (the base) 0.449 0.439 0.441 0.544 0.755
au-property-embed-v1 0.679 0.708 0.676 0.781 0.709
au-property-embed-v2 0.696 0.722 0.737 0.778 0.751
  • In-distribution: new listings and queries from the same suburbs and writer as training.
  • Gold: 200 queries written one at a time by the AI assistant against the test corpus.
  • Unseen suburbs: queries about suburbs held out of training.
  • Unseen writer: queries about listings written by a different model with different style cards.
  • General: mean over five NanoBEIR sets (MS MARCO, NQ, FiQA, SciFact, Quora), 50 queries each.

All four property suites search one shared test corpus. Every gain over the base has a 95% paired bootstrap interval well clear of zero. Recall@100 is 0.92 to 0.96 on the property suites, against 0.73 to 0.79 for the base.

Held-out documents only. Searching only the held-out documents (706 and 868), v2 scores 0.823 on unseen suburbs and 0.842 on the unseen writer, against 0.540 and 0.577 for the base and 0.601 and 0.629 for BM25.

Success criteria. All four criteria set before training were met: gains over the base on in-distribution (+0.248), gold (+0.283) and unseen writer (+0.233) with intervals above zero, and a general-retrieval loss of no more than 0.02 (it lost 0.003).

Smaller embeddings

Variant In-distribution Gold Unseen suburbs Unseen writer
768 dimensions 0.696 0.722 0.737 0.778
int8 embeddings 0.696 0.721 0.736 0.778
512 dimensions 0.697 0.712 0.727 0.767
256 dimensions 0.672 0.695 0.716 0.754
128 dimensions 0.625 0.662 0.691 0.720
64 dimensions 0.566 0.607 0.639 0.636
Binary embeddings 0.658 0.686 0.696 0.747

Even 64 dimensions beats the full-size base model on every property suite.

Strengths and weaknesses

  • Strong: placing suburbs in a wider area (city, compass sector, state), including suburbs never seen in training; separating same-name suburbs in different states; architectural styles (Queenslander, terrace), features, agent profiles, and rent versus sale.
  • Weak: "near X" and suburb-profile queries where the answer depends on geography not in the text; postcodes (only at BM25's level); negation (fails completely); and on minimal-pair tests, budgets and regions, where v2 is worse than v1.

Training

Training data

No commercial, employer or portal data was used. All external sources are public and ungated.

Source Licence Used for
Alibaba-NLP/gte-modernbert-base Apache-2.0 Base model
dylanhogg/gnaf-2022 (G-NAF, the government's open national address file) CC BY 4.0, with the G-NAF end-user terms Street names for invented addresses
blake-resic/australian-suburbs-postcodes CC BY 4.0 Suburb, state, postcode, nearest city and distance
wikimedia/wikipedia, English, 2023-11-01 CC BY-SA 3.0 and GFDL Suburb article leads used as documents (the only real text), and amenity evidence per suburb
sentence-transformers/gooaq Upstream GooAQ: Apache-2.0 2,166 general question-answer pairs
sentence-transformers/squad Upstream SQuAD: CC BY-SA 4.0 1,834 general question-answer pairs
NanoBEIR: MS MARCO, NQ, FiQA, SciFact, Quora (zeta-alpha-ai) CC BY 4.0 as published Evaluation only, never trained on

Licences are as declared on each source's Hugging Face page or its upstream source, checked on 2026-11-12. Wikipedia content is by Wikipedia contributors, and G-NAF is © Geoscape Australia, licensed by the Commonwealth of Australia under the Open G-NAF End User Licence Agreement.

Written for this project:

Data How it was made
3,958 listings and agent profiles Written by Claude models from records sampled by code. Addresses combine a real street name and suburb with an invented number. Every person and agency is invented.
3,724 search queries Written by Claude models from sampled query specifications.
7,248 template queries Rendered by code from the same specifications (training only).
Glossary, abbreviation and state-dialect tables Written by the AI assistant from general knowledge. No document was copied.
Prices and rents A simple model in code (city median by ring and property type), not market data.
Relevance labels Computed from the sampled records, not annotated.

Procedure

  • Rows: 15,290 in total: 9,622 search triplets with hard negatives, 1,668 term pairs (abbreviations, synonyms, dialect) and 4,000 general question-answer pairs (26% of rows).
  • Loss: Matryoshka loss (768, 512, 256, 128, 64) over cached multiple-negatives ranking loss.
  • Settings: 2 epochs, contrastive batch 64 (mini-batch 16), learning rate 1e-5 with 10% warm-up and cosine decay, inputs truncated to 256 tokens, no frozen layers, seed 0.
  • Precision: full precision during training, saved in half precision as the base ships.
  • Checkpoint: the step-420 checkpoint (of 476) was kept on validation nDCG@10 (0.716).
  • Hardware: an Apple M1 Max laptop GPU (PyTorch MPS backend), 50 minutes.

The full settings are in train_config.json.

Limitations

  • Trained and tested on synthetic text. Listings, agent profiles and queries were written by language models from sampled records. Only the suburb articles (Wikipedia) are real. Documents, queries and labels come from the same record sampler and vocabulary, so this is a closed loop. Scores on real listing data will differ.
  • The test suites have been used for three models (v0, v1 and this one), and v2 was chosen as the release after its test scores were seen. Nothing was tuned on test scores, but the suites are no longer untouched.
  • The test probably favours the finetuned model. 44% of the test corpus is location-swapped copies of other documents, a construct also used for some training negatives. The baselines never saw it, so the gain over them is likely overstated by an unknown amount.
  • One training run. There is no measure of run-to-run variance. Intervals cover query sampling only.
  • The general check is small (five sets of 50 queries) and may slightly favour v2, because SQuAD and NQ are both question answering over Wikipedia.
  • English only, Australian property only. Behaviour on other countries' property text was not tested.

More information

  • Previous versions: v0 (property data only) lost most of its general retrieval ability. v1 recovered most of it by blending v0's weights with the base. v2 replaces both.

Citation

@misc{hogg2026aupropertyembed,
  title        = {au-property-embed-v2: a text embedding model for Australian property search},
  author       = {Hogg, Dylan},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/dylanhogg/au-property-embed-v2}}
}

Please also cite the base model, Alibaba-NLP/gte-modernbert-base.

Downloads last month
13
Safetensors
Model size
0.1B params
Tensor type
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for dylanhogg/au-property-embed-v2

Quantized
(16)
this model

Datasets used to train dylanhogg/au-property-embed-v2

Evaluation results

  • ndcg_at_10 on AU property search, in-distribution (500 queries)
    self-reported
    0.696
  • ndcg_at_10 on AU property search, gold (200 queries)
    self-reported
    0.722
  • ndcg_at_10 on AU property search, unseen suburbs (300 queries)
    self-reported
    0.737
  • ndcg_at_10 on AU property search, unseen writer (350 queries)
    self-reported
    0.778
  • ndcg_at_10 on NanoBEIR mean of MS MARCO, NQ, FiQA, SciFact, Quora
    self-reported
    0.751