Skip to content

DBSCAN implemented (and incanter facilities in HiKM corrected) #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

DBSCAN implemented (and incanter facilities in HiKM corrected) #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 34 additions & 2 deletions 04-Clustering/clojure-ajw/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ [email protected]

1. an implementation of k-means clustering in haskell
2. an implementation of hierarchical k-menas clustering in clojure
3. an implementation of DBSCAN clustering in clojure

---------------------
1. k-means clustering
Expand Down Expand Up @@ -50,8 +51,8 @@ It prints out the results.

The results are anomalous.

> (hc/fisher-cluster)
> (hc/ajw-cluster)
> (hc/fisher-pca)
> (hc/hi-pca)

print out 2D principal component incanter charts to check what is goin on.

Expand All @@ -62,4 +63,35 @@ The incanter code is based on:
The relevant code is in the namespace hierarchical.
The (imperfect) clustering is decided in 'closest-vectors'.

--------------------
3. DBSCAN clustering
--------------------

This is developed from the code on git: NeoTeo/DBScan
amended to deal with vectors rather than single values

From core:

(show-distances) and
(show-regions distance)

generate incanter histograms indicating the distances between points and the
number of points assocaited with each point given a specified distance.

Choosing eps and minPts On the basis of this information:

(iris-db-cluster eps minpts) generates an incanter pricipal components 2D scatter plot of
the resulting clustering.

(iris-db-cluster) defaults to (iris-db-cluster 0.5 15)


(db/show-distances))

; histogram

(defn show-regions [dist]
(db/show-regions dist))



38 changes: 35 additions & 3 deletions 04-Clustering/clojure-ajw/README.md~
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ [email protected]

1. an implementation of k-means clustering in haskell
2. an implementation of hierarchical k-menas clustering in clojure
3. an implementation of DBScan clustering in clojure

---------------------
1. k-means clustering
Expand All @@ -13,7 +14,7 @@ The code is developed from the demo code in

Mena: Beginning Haskell - Chapter 6: Data Mining

The code runs straightforwardly in Leksah
The code runs straightforwardly in the GUI Leksah

(which is quite adequate in my experience
- better that web reviews suggest -
Expand Down Expand Up @@ -50,8 +51,8 @@ It prints out the results.

The results are anomalous.

> (hc/fisher-cluster)
> (hc/ajw-cluster)
> (hc/fisher-pca)
> (hc/hi-pca)

print out 2D principal component incanter charts to check what is goin on.

Expand All @@ -62,4 +63,35 @@ The incanter code is based on:
The relevant code is in the namespace hierarchical.
The (imperfect) clustering is decided in 'closest-vectors'.

--------------------
3. DBSCAN clustering
--------------------

This is developed from the code on git: NeoTeo/DBScan
amended to deal with vectors rather than single values

From core:

(show-distances) and
(show-regions distance)

generate incanter histograms indicating the distances between points and the
number of points assocaited with each point given a specified distance.

Choosing eps and minPts On the basis of this information:

(iris-db-cluster eps minpts) generates an incanter pricipal components 2D scatter plot of
the resulting clustering.

(iris-db-cluster) defaults to (iris-db-cluster 0.5 15)


(db/show-distances))

; histogram

(defn show-regions [dist]
(db/show-regions dist))



878 changes: 878 additions & 0 deletions 04-Clustering/clojure-ajw/clojure/hs_err_pid24138.log
View file
Open in desktop

Large diffs are not rendered by default.

150 changes: 150 additions & 0 deletions 04-Clustering/clojure-ajw/clojure/resources/iris-wiki-incanter.data
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
5.1,3.5,1.4,0.2,setosa
4.9,3.0,1.4,0.2,setosa
4.7,3.2,1.3,0.2,setosa
4.6,3.1,1.5,0.2,setosa
5.0,3.6,1.4,0.2,setosa
5.4,3.9,1.7,0.4,setosa
4.6,3.4,1.4,0.3,setosa
5.0,3.4,1.5,0.2,setosa
4.4,2.9,1.4,0.2,setosa
4.9,3.1,1.5,0.1,setosa
5.4,3.7,1.5,0.2,setosa
4.8,3.4,1.6,0.2,setosa
4.8,3.0,1.4,0.1,setosa
4.3,3.0,1.1,0.1,setosa
5.8,4.0,1.2,0.2,setosa
5.7,4.4,1.5,0.4,setosa
5.4,3.9,1.3,0.4,setosa
5.1,3.5,1.4,0.3,setosa
5.7,3.8,1.7,0.3,setosa
5.1,3.8,1.5,0.3,setosa
5.4,3.4,1.7,0.2,setosa
5.1,3.7,1.5,0.4,setosa
4.6,3.6,1.0,0.2,setosa
5.1,3.3,1.7,0.5,setosa
4.8,3.4,1.9,0.2,setosa
5.0,3.0,1.6,0.2,setosa
5.0,3.4,1.6,0.4,setosa
5.2,3.5,1.5,0.2,setosa
5.2,3.4,1.4,0.2,setosa
4.7,3.2,1.6,0.2,setosa
4.8,3.1,1.6,0.2,setosa
5.4,3.4,1.5,0.4,setosa
5.2,4.1,1.5,0.1,setosa
5.5,4.2,1.4,0.2,setosa
4.9,3.1,1.5,0.2,setosa
5.0,3.2,1.2,0.2,setosa
5.5,3.5,1.3,0.2,setosa
4.9,3.6,1.4,0.1,setosa
4.4,3.0,1.3,0.2,setosa
5.1,3.4,1.5,0.2,setosa
5.0,3.5,1.3,0.3,setosa
4.5,2.3,1.3,0.3,setosa
4.4,3.2,1.3,0.2,setosa
5.0,3.5,1.6,0.6,setosa
5.1,3.8,1.9,0.4,setosa
4.8,3.0,1.4,0.3,setosa
5.1,3.8,1.6,0.2,setosa
4.6,3.2,1.4,0.2,setosa
5.3,3.7,1.5,0.2,setosa
5.0,3.3,1.4,0.2,setosa
7.0,3.2,4.7,1.4,versicolor
6.4,3.2,4.5,1.5,versicolor
6.9,3.1,4.9,1.5,versicolor
5.5,2.3,4.0,1.3,versicolor
6.5,2.8,4.6,1.5,versicolor
5.7,2.8,4.5,1.3,versicolor
6.3,3.3,4.7,1.6,versicolor
4.9,2.4,3.3,1.0,versicolor
6.6,2.9,4.6,1.3,versicolor
5.2,2.7,3.9,1.4,versicolor
5.0,2.0,3.5,1.0,versicolor
5.9,3.0,4.2,1.5,versicolor
6.0,2.2,4.0,1.0,versicolor
6.1,2.9,4.7,1.4,versicolor
5.6,2.9,3.6,1.3,versicolor
6.7,3.1,4.4,1.4,versicolor
5.6,3.0,4.5,1.5,versicolor
5.8,2.7,4.1,1.0,versicolor
6.2,2.2,4.5,1.5,versicolor
5.6,2.5,3.9,1.1,versicolor
5.9,3.2,4.8,1.8,versicolor
6.1,2.8,4.0,1.3,versicolor
6.3,2.5,4.9,1.5,versicolor
6.1,2.8,4.7,1.2,versicolor
6.4,2.9,4.3,1.3,versicolor
6.6,3.0,4.4,1.4,versicolor
6.8,2.8,4.8,1.4,versicolor
6.7,3.0,5.0,1.7,versicolor
6.0,2.9,4.5,1.5,versicolor
5.7,2.6,3.5,1.0,versicolor
5.5,2.4,3.8,1.1,versicolor
5.5,2.4,3.7,1.0,versicolor
5.8,2.7,3.9,1.2,versicolor
6.0,2.7,5.1,1.6,versicolor
5.4,3.0,4.5,1.5,versicolor
6.0,3.4,4.5,1.6,versicolor
6.7,3.1,4.7,1.5,versicolor
6.3,2.3,4.4,1.3,versicolor
5.6,3.0,4.1,1.3,versicolor
5.5,2.5,4.0,1.3,versicolor
5.5,2.6,4.4,1.2,versicolor
6.1,3.0,4.6,1.4,versicolor
5.8,2.6,4.0,1.2,versicolor
5.0,2.3,3.3,1.0,versicolor
5.6,2.7,4.2,1.3,versicolor
5.7,3.0,4.2,1.2,versicolor
5.7,2.9,4.2,1.3,versicolor
6.2,2.9,4.3,1.3,versicolor
5.1,2.5,3.0,1.1,versicolor
5.7,2.8,4.1,1.3,versicolor
6.3,3.3,6.0,2.5,virginica
5.8,2.7,5.1,1.9,virginica
7.1,3.0,5.9,2.1,virginica
6.3,2.9,5.6,1.8,virginica
6.5,3.0,5.8,2.2,virginica
7.6,3.0,6.6,2.1,virginica
4.9,2.5,4.5,1.7,virginica
7.3,2.9,6.3,1.8,virginica
6.7,2.5,5.8,1.8,virginica
7.2,3.6,6.1,2.5,virginica
6.5,3.2,5.1,2.0,virginica
6.4,2.7,5.3,1.9,virginica
6.8,3.0,5.5,2.1,virginica
5.7,2.5,5.0,2.0,virginica
5.8,2.8,5.1,2.4,virginica
6.4,3.2,5.3,2.3,virginica
6.5,3.0,5.5,1.8,virginica
7.7,3.8,6.7,2.2,virginica
7.7,2.6,6.9,2.3,virginica
6.0,2.2,5.0,1.5,virginica
6.9,3.2,5.7,2.3,virginica
5.6,2.8,4.9,2.0,virginica
7.7,2.8,6.7,2.0,virginica
6.3,2.7,4.9,1.8,virginica
6.7,3.3,5.7,2.1,virginica
7.2,3.2,6.0,1.8,virginica
6.2,2.8,4.8,1.8,virginica
6.1,3.0,4.9,1.8,virginica
6.4,2.8,5.6,2.1,virginica
7.2,3.0,5.8,1.6,virginica
7.4,2.8,6.1,1.9,virginica
7.9,3.8,6.4,2.0,virginica
6.4,2.8,5.6,2.2,virginica
6.3,2.8,5.1,1.5,virginica
6.1,2.6,5.6,1.4,virginica
7.7,3.0,6.1,2.3,virginica
6.3,3.4,5.6,2.4,virginica
6.4,3.1,5.5,1.8,virginica
6.0,3.0,4.8,1.8,virginica
6.9,3.1,5.4,2.1,virginica
6.7,3.1,5.6,2.4,virginica
6.9,3.1,5.1,2.3,virginica
5.8,2.7,5.3,1.9,virginica
6.8,3.2,5.9,2.3,virginica
6.7,3.3,5.7,2.5,virginica
6.7,3.0,5.2,2.3,virginica
6.3,2.5,5.0,1.9,virginica
6.5,3.0,5.2,2.0,virginica
6.2,3.4,5.4,2.3,virginica
5.9,3.0,5.1,1.8,virginica
2 changes: 1 addition & 1 deletion 04-Clustering/clojure-ajw/clojure/resources/iris.data
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@
6.9,3.1,5.4,2.1,virginica
6.7,3.1,5.6,2.4,virginica
6.9,3.1,5.1,2.3,virginica
5.8,2.7,5.1,1.9,virginica
5.8,2.7,5.3,1.9,virginica
6.8,3.2,5.9,2.3,virginica
6.7,3.3,5.7,2.5,virginica
6.7,3.0,5.2,2.3,virginica
Expand Down
49 changes: 41 additions & 8 deletions 04-Clustering/clojure-ajw/clojure/src/clj_ml7/core.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,56 @@
[clj-ml7.som :as som]
[clj-ml7.pca :as pca]
[clj-ml7.incanter :as inc]


)
[clj-ml7.dbscan :as db])

(:require [incanter.core :as ic]
[incanter.datasets :as id]
[clj-ml.data :as cd]
[clj-ml.clusterers :as cc])

)
[incanter.datasets :as id]
[clj-ml.data :as cd]
[clj-ml.clusterers :as cc])
)


;; ===
;; AJW
;; ===

;; ---------------
;; 1. hierarchical
;; ---------------

(defn iris-hi-cluster [n]
(hc/iris-cluster-join n))

; results

(defn fisher-pca []
(hc/fisher-pca))

(defn hi-pca []
(hc/hi-pca))


;; ---------
;; 2. dbscan
;; ---------

; histogram of distances between points

(defn show-distances []
(db/show-distances))

; histogram

(defn show-regions [dist]
(db/show-regions dist))

; demo

(defn iris-db-cluster
([] (iris-db-cluster 0.5 15))
([eps min-pts] (db/iris-pca (db/prepare-pca (db/dbscan-iris eps min-pts)))))


;; ==========
;; AKHIL WALI
;; ==========
Expand Down Expand Up @@ -190,4 +221,6 @@
(defn km [] (ns clj-ml7.kmeans))
(defn hi [] (ns clj-ml7.hierarchical))
(defn incanter [] (ns clj-ml7.incanter))
(defn pca [] (ns clj-ml7.pca))
(defn db [] (ns clj-ml7.dbscan))

Loading