p/gallery
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: p:05c3687ab15b6236533ef88fb1c8cffd79884126
Branches Tags
p:master
..
compare: p:1e3800cc16ab9c69ae40e11f4374d76ef0350aa0
Branches Tags
p:master

No commits in common. "05c3687ab15b6236533ef88fb1c8cffd79884126" and "1e3800cc16ab9c69ae40e11f4374d76ef0350aa0" have entirely different histories.
05c3687ab1 ... 1e3800cc16

3 changed files with 16 additions and 30 deletions
Show Stats Expand all files Collapse all files

7
README.adoc → README
View File

@ -1,6 +1,3 @@
gallery
=======
This is gallery software designed to maintain a shadow structure
of your filesystem, in which you can attach metadata to your media,
and query your collections in various ways.
@ -8,10 +5,10 @@ and query your collections in various ways.
All media is content-addressed by its SHA-1 hash value, and at your option
also perceptually hashed. Duplicate search is an essential feature.
Prerequisites: Go, ImageMagick v7, xdg-utils
Prerequisites: Go, ImageMagick, xdg-utils
The gallery is designed for simplicity, and easy interoperability.
sqlite3, curl, jq, and the filesystem will take you a long way.
The intended mode of use is running daily automated sync/thumbnail/dhash/tag
batches in a cron job, or from a system timer. See _test.sh_ for usage hints.
batches in a cron job, or from a system timer. See test.sh for usage hints.

37
deeptagger/README.adoc
View File

@ -47,18 +47,7 @@ Options
--pipe::
Take input filenames from the standard input.
--threshold 0.1::
Output weight threshold. Needs to be set higher on ML-Danbooru models.
Tagging galleries
-----------------
The appropriate invocation depends on your machine, and the chosen model.
Unless you have a powerful machine, or use a fast model, it may take forever.
$ find "$GALLERY/images" -type f \
| build/deeptagger --pipe -b 16 -t 0.5 \
models/ml_caformer_m36_dec-5-97527.model \
| sed 's|[^\t]*/||' \
| gallery tag "$GALLERY" caformer "ML-Danbooru CAFormer"
Output weight threshold. Needs to be set very high on ML-Danbooru models.
Model benchmarks (Linux)
------------------------
@ -76,14 +65,14 @@ GPU inference
|WD v1.4 ViT v2 (batch)|16|19 s
|DeepDanbooru|16|21 s
|WD v1.4 SwinV2 v2 (batch)|16|21 s
|ML-Danbooru CAFormer dec-5-97527|16|25 s
|ML-Danbooru Caformer dec-5-97527|16|25 s
|WD v1.4 ViT v2 (batch)|4|27 s
|WD v1.4 SwinV2 v2 (batch)|4|30 s
|DeepDanbooru|4|31 s
|ML-Danbooru TResNet-D 6-30000|16|31 s
|WD v1.4 MOAT v2 (batch)|16|31 s
|WD v1.4 ConvNeXT v2 (batch)|16|32 s
|ML-Danbooru CAFormer dec-5-97527|4|32 s
|ML-Danbooru Caformer dec-5-97527|4|32 s
|WD v1.4 ConvNeXTV2 v2 (batch)|16|36 s
|ML-Danbooru TResNet-D 6-30000|4|39 s
|WD v1.4 ConvNeXT v2 (batch)|4|39 s
@ -91,7 +80,7 @@ GPU inference
|WD v1.4 ConvNeXTV2 v2 (batch)|4|43 s
|WD v1.4 ViT v2|1|43 s
|WD v1.4 ViT v2 (batch)|1|43 s
|ML-Danbooru CAFormer dec-5-97527|1|52 s
|ML-Danbooru Caformer dec-5-97527|1|52 s
|DeepDanbooru|1|53 s
|WD v1.4 MOAT v2|1|53 s
|WD v1.4 ConvNeXT v2|1|54 s
@ -121,7 +110,7 @@ CPU inference
|WD v1.4 ConvNeXTV2 v2|1|245 s
|WD v1.4 ConvNeXTV2 v2 (batch)|4|268 s
|WD v1.4 ViT v2 (batch)|16|270 s
|ML-Danbooru CAFormer dec-5-97527|4|270 s
|ML-Danbooru Caformer dec-5-97527|4|270 s
|WD v1.4 ConvNeXT v2 (batch)|1|272 s
|WD v1.4 SwinV2 v2 (batch)|4|277 s
|WD v1.4 ViT v2 (batch)|4|277 s
@ -129,7 +118,7 @@ CPU inference
|WD v1.4 SwinV2 v2 (batch)|1|300 s
|WD v1.4 SwinV2 v2|1|302 s
|WD v1.4 SwinV2 v2 (batch)|16|305 s
|ML-Danbooru CAFormer dec-5-97527|16|305 s
|ML-Danbooru Caformer dec-5-97527|16|305 s
|WD v1.4 MOAT v2 (batch)|4|307 s
|WD v1.4 ViT v2|1|308 s
|WD v1.4 ViT v2 (batch)|1|311 s
@ -137,7 +126,7 @@ CPU inference
|WD v1.4 MOAT v2|1|332 s
|WD v1.4 MOAT v2 (batch)|16|335 s
|WD v1.4 MOAT v2 (batch)|1|339 s
|ML-Danbooru CAFormer dec-5-97527|1|352 s
|ML-Danbooru Caformer dec-5-97527|1|352 s
|===
Model benchmarks (macOS)
@ -177,12 +166,12 @@ GPU inference
|WD v1.4 ConvNeXTV2 v2 (batch)|1|160 s
|WD v1.4 MOAT v2 (batch)|1|165 s
|WD v1.4 SwinV2 v2|1|166 s
|ML-Danbooru CAFormer dec-5-97527|1|263 s
|ML-Danbooru Caformer dec-5-97527|1|263 s
|WD v1.4 ConvNeXT v2|1|273 s
|WD v1.4 MOAT v2|1|273 s
|WD v1.4 ConvNeXTV2 v2|1|340 s
|ML-Danbooru CAFormer dec-5-97527|4|445 s
|ML-Danbooru CAFormer dec-5-97527|8|1790 s
|ML-Danbooru Caformer dec-5-97527|4|445 s
|ML-Danbooru Caformer dec-5-97527|8|1790 s
|WD v1.4 MOAT v2 (batch)|4|kernel panic
|===
@ -200,14 +189,14 @@ CPU inference
|WD v1.4 SwinV2 v2 (batch)|1|98 s
|ML-Danbooru TResNet-D 6-30000|4|99 s
|WD v1.4 SwinV2 v2|1|99 s
|ML-Danbooru CAFormer dec-5-97527|4|110 s
|ML-Danbooru CAFormer dec-5-97527|8|110 s
|ML-Danbooru Caformer dec-5-97527|4|110 s
|ML-Danbooru Caformer dec-5-97527|8|110 s
|WD v1.4 ViT v2 (batch)|4|111 s
|WD v1.4 ViT v2 (batch)|8|111 s
|WD v1.4 ViT v2 (batch)|1|113 s
|WD v1.4 ViT v2|1|113 s
|ML-Danbooru TResNet-D 6-30000|1|118 s
|ML-Danbooru CAFormer dec-5-97527|1|122 s
|ML-Danbooru Caformer dec-5-97527|1|122 s
|WD v1.4 ConvNeXT v2 (batch)|8|124 s
|WD v1.4 ConvNeXT v2 (batch)|4|125 s
|WD v1.4 ConvNeXTV2 v2 (batch)|8|129 s

2
deeptagger/download.sh
View File

@ -157,7 +157,7 @@ wd14 'WD v1.4 SwinV2 v2' 'SmilingWolf/wd-v1-4-swinv2-tagger-v2'
wd14 'WD v1.4 MOAT v2' 'SmilingWolf/wd-v1-4-moat-tagger-v2'
# As suggested by author https://github.com/IrisRainbowNeko/ML-Danbooru-webui
mldanbooru 'ML-Danbooru CAFormer dec-5-97527' \
mldanbooru 'ML-Danbooru Caformer dec-5-97527' \
448 'ml_caformer_m36_dec-5-97527.onnx'
mldanbooru 'ML-Danbooru TResNet-D 6-30000' \
640 'TResnet-D-FLq_ema_6-30000.onnx'