software

ANNchor – Accelerating k-NN Graphs For Complex Metrics With Machine Learning

ANNchor is a python library which constructs approximate k-nearest neighbour graphs for slow metrics.

The k-NN graph is an extremely useful data structure that appears in a wide variety of applications, for example: clustering, dimensionality reduction, visualisation and exploratory data analysis (EDA).

However, if we want to use a slow metric, these k-NN graphs can take an exceptionally long time to compute.

Typical slow metrics include the Wasserstein metric (Earth Mover’s distance) applied to images, and Levenshtein (Edit) distance on long strings, where the time taken to compute these distances is significantly longer than a typical Euclidean distance.

ANNchor uses Machine Learning methods to infer true distances between points in a data set from a variety of features derived from anchor points (aka landmarks/waypoints).

In practice, this means that ANNchor does not make as many calls to the underlying metric as other state of the art k-NN graph generation techniques.

This translates to quicker run times, especially when the metric is slow.

Results from ANNchor can easily be combined with other popular libraries in the Data Science community.

In the docs we give examples of how to use ANNchor in an EDA pipeline alongside UMAP and HDBSCAN.

Installation

Clone this repo and install with pip:

pip install git+https://github.com/gchq/annchor.git

Basic Usage

import numpy as np
import annchor

X =          #your data, list/np.array of items
distance =   #your distance function, distance(X[i],X[j]) = d

ann = annchor.Annchor(X,
                      distance,
                      n_anchors=15,
                      n_neighbors=15,
                      p_work=0.1)
ann.fit()

print(ann.neighbor_graph)
Varshini

Varshini is a Cyber Security expert in Threat Analysis, Vulnerability Assessment, and Research. Passionate about staying ahead of emerging Threats and Technologies.

Recent Posts

JBDev : A Tool For Jailbreak And TrollStore Development

JBDev is a specialized development tool designed to streamline the creation and debugging of jailbreak…

7 hours ago

Kereva LLM Code Scanner : A Revolutionary Tool For Python Applications Using LLMs

The Kereva LLM Code Scanner is an innovative static analysis tool tailored for Python applications…

8 hours ago

Nuclei-Templates-Labs : A Hands-On Security Testing Playground

Nuclei-Templates-Labs is a dynamic and comprehensive repository designed for security researchers, learners, and organizations to…

10 hours ago

SSH-Stealer : The Stealthy Threat Of Advanced Credential Theft

SSH-Stealer and RunAs-Stealer are malicious tools designed to stealthily harvest SSH credentials, enabling attackers to…

10 hours ago

ollvm-unflattener : A Tool For Reversing Control Flow Flattening In OLLVM

Control flow flattening is a common obfuscation technique used by OLLVM (Obfuscator-LLVM) to transform executable…

10 hours ago

Cybersecurity – Tools And Their Function

Cybersecurity tools play a critical role in safeguarding digital assets, systems, and networks from malicious…

1 day ago