Queryable

# Queryable

The open-source code of **Queryable**, an offline photo search app for iPhone and iPad (iOS 18 or later). Describe a photo in your own words, such as `a brown dog sitting on a bench`, and Apple's [MobileCLIP](https://github.com/apple/ml-mobileclip) model finds it on the device. Your photos are never uploaded.

<a href="https://apps.apple.com/app/apple-store/id1661598353?pt=128849838&ct=github-readme&mt=8">
    <img src="https://github-production-user-asset-6210df.s3.amazonaws.com/6824141/252914927-51414112-236b-4f7a-a13b-5210f9203198.svg" alt="Download on the App Store">
</a>

[![Queryable](https://mazzzystar.com/images/2022-12-28/Queryable-search-result.jpg)](https://apps.apple.com/app/apple-store/id1661598353?pt=128849838&ct=github-readme&mt=8)

[Blog](https://mazzzystar.com/2022/12/29/Run-CLIP-on-iPhone-to-Search-Photos/) | [App Store](https://apps.apple.com/app/apple-store/id1661598353?pt=128849838&ct=github-readme&mt=8) | [Website](https://queryable.app/) | [Story](https://mazzzystar.com/2024/07/21/Two-Years-of-an-AI-Photo-Album-Search-App/) | [故事](https://mazzzystar.com/2024/07/21/Two-Years-of-an-AI-Photo-Album-Search-App-zh/)

## How does it work?

- Encode all album photos using the CLIP Image Encoder, compute image vectors, and save them.
- For each new text query, compute the corresponding text vector using the Text Encoder.
- Compare the similarity between this text vector and each image vector.
- Rank and return the top K most similar results.

The process is as follows:

![](https://raw.githubusercontent.com/mazzzystar/Queryable/ce184131123650fb014eaa8514e37b1202625d14/Queryable/Queryable/Assets.xcassets/Queryable-flow-chart.jpeg)

For more details, please refer to my blog: [Run CLIP on iPhone to Search Photos](https://mazzzystar.com/2022/12/29/Run-CLIP-on-iPhone-to-Search-Photos/).

# Updates

[2024-09-01]: Now supports Apple's [MobileCLIP](https://github.com/apple/ml-mobileclip).

You can download the exported `TextEncoder_mobileCLIP_s2.mlmodelc` and `ImageEncoder_mobileCLIP_s2.mlmodelc` from [Google Drive](https://drive.google.com/drive/folders/12ze3UcqrXt9qeySGh_j_zWE-PWRDTzJv?usp=drive_link). Currently we use `s2` model as the default model, which balances both efficiency & precision.

## [PicQuery](https://github.com/greyovo/PicQuery)(Android)

<a href="https://play.google.com/store/apps/details?id=me.grey.picquery">
    <img src="https://github-production-user-asset-6210df.s3.amazonaws.com/6824141/274861421-69a37ae7-55b3-46b2-ad24-5368eb2734f9.png" alt="download-on-the-app-store" width="120">
</a>

The Android version([Code](https://github.com/greyovo/PicQuery)) developed by [@greyovo](https://github.com/greyovo), which supports both English and Chinese. See details in [#12](https://github.com/mazzzystar/Queryable/issues/12).

## Run on Xcode

Download the `TextEncoder_mobileCLIP_s2.mlmodelc` and `ImageEncoder_mobileCLIP_s2.mlmodelc` from [Google Drive](https://drive.google.com/drive/folders/12ze3UcqrXt9qeySGh_j_z