Commit Graph
39 Commits
Author SHA1 Message Date
aria1th 13445738d9 Fix tensorboard related functions 2023-01-16 03:02:54 +09:00
aria1th 598f7fcd84 Fix loss_dict problem 2023-01-16 02:46:21 +09:00
AngelBottomlessandGitHub 16f410893e fix missing 'mean loss' for tensorboard integration 2023-01-16 02:08:47 +09:00
aria1th a4a5475cfa Variable dropout rate
Implements variable dropout rate from #4549

Fixes hypernetwork multiplier being able to modified during training, also fixes user-errors by setting multiplier value to lower values for training.

Changes function name to match torch.nn.module standard

Fixes RNG reset issue when generating previews by restoring RNG state
2023-01-10 14:56:57 +09:00
AngelBottomlessandGitHub 467d8b967b Fix errors from commit f2b697 with --hide-ui-dir-config
https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/f2b69709eaff88fc3a2bd49585556ec0883bf5ea
2022-11-05 04:24:42 +09:00
AngelBottomlessandGitHub 7278897982 Update shared.py 2022-11-04 17:12:28 +09:00
aria1th 1ca0bcd3a7 only save if option is enabled 2022-11-04 16:09:19 +09:00
aria1th f5d394214d split before declaring file name 2022-11-04 16:04:03 +09:00
aria1th 283249d239 apply 2022-11-04 15:57:17 +09:00
AngelBottomlessandGitHub 179702adc4 Merge branch 'AUTOMATIC1111:master' into force-push-patch-13 2022-11-04 15:51:09 +09:00
AngelBottomlessandGitHub 0d07cbfa15 I blame code autocomplete 2022-11-04 15:50:54 +09:00
aria1th 0abb39f461 resolve conflict - first revert 2022-11-04 15:47:19 +09:00
aria1th 1764ac3c8b use hash to check valid optim 2022-11-03 14:49:26 +09:00
aria1th 0b143c1163 Separate .optim file from model 2022-11-03 14:30:53 +09:00
AngelBottomlessandGitHub 7ea5956ad5 now add 2022-11-02 22:18:55 +09:00
AngelBottomlessandGitHub 10b280e9a2 Merge branch 'AUTOMATIC1111:master' into force-push-patch-13 2022-11-02 22:18:31 +09:00
AngelBottomlessandGitHub 9b5f85ac83 first revert 2022-11-02 22:18:04 +09:00
AngelBottomlessandGitHub 3178c35224 resolve conflicts 2022-11-02 22:16:32 +09:00
aria1th 9d96d7d0a0 resolve conflicts 2022-10-30 20:40:59 +09:00
AngelBottomless 20194fd975 We have duplicate linear now 2022-10-30 20:40:59 +09:00
AngelBottomless 4b8a192f68 add optimizer save option to shared.opts 2022-10-30 20:40:59 +09:00
AngelBottomlessandGitHub f361e804eb Re enable linear 2022-10-29 08:36:50 +09:00
AngelBottomlessandGitHub 462e6ba667 Disable unavailable or duplicate options 2022-10-27 15:40:24 +09:00
AngelBottomlessandGitHub 029d7c7543 Revert unresolved changes in Bias initialization
it should be zeros_ or parameterized in future properly.
2022-10-27 14:44:53 +09:00
AngelBottomless 85fcccc105 Squashed commit of fixing dropout silently
fix dropouts for future hypernetworks

add kwargs for Hypernetwork class

hypernet UI for gradio input

add recommended options

remove as options

revert adding options in ui
2022-10-27 14:38:21 +09:00
AngelBottomlessandAUTOMATIC1111 7207e3bf49 remove duplicate keys and lowercase 2022-10-26 09:17:01 +03:00
AngelBottomlessandAUTOMATIC1111 de096d0ce7 Weight initialization and More activation func
add weight init

add weight init option in create_hypernetwork

fstringify hypernet info

save weight initialization info for further debugging

fill bias with zero for He/Xavier

initialize LayerNorm with Normal

fix loading weight_init
2022-10-26 09:17:01 +03:00
AngelBottomlessandAUTOMATIC1111 e9a410b535 check length for variance 2022-10-24 09:07:39 +03:00
AngelBottomlessandAUTOMATIC1111 0d2e1dac40 convert deque -> list
I don't feel this being efficient
2022-10-24 09:07:39 +03:00
AngelBottomlessandAUTOMATIC1111 348f89c8d4 statistics for pbar 2022-10-24 09:07:39 +03:00
AngelBottomlessandAUTOMATIC1111 40b56c9289 cleanup some code 2022-10-24 09:07:39 +03:00
AngelBottomlessandAUTOMATIC1111 b297cc3324 Hypernetworks - fix KeyError in statistics caching
Statistics logging has changed to {filename : list[losses]}, so it has to use loss_info[key].pop()
2022-10-24 09:07:39 +03:00
AngelBottomlessandAUTOMATIC1111 48dbf99e84 Allow tracking real-time loss
Someone had 6000 images in their dataset, and it was shown as 0, which was confusing.
This will allow tracking real time dataset-average loss for registered objects.
2022-10-22 22:24:19 +03:00
AngelBottomlessandAUTOMATIC1111 24694e5983 Update hypernetwork.py 2022-10-22 20:25:32 +03:00
aria1th f89829ec3a Revert "fix bugs and optimizations"
This reverts commit 108be15500.
2022-10-21 01:37:11 +09:00
AngelBottomlessandGitHub 108be15500 fix bugs and optimizations 2022-10-21 01:00:41 +09:00
AngelBottomlessandGitHub a71e021236 only linear 2022-10-20 23:48:52 +09:00
AngelBottomlessandGitHub d8acd34f66 generalized some functions and option for ignoring first layer 2022-10-20 23:43:03 +09:00
AngelBottomlessandAUTOMATIC1111 703e6d9e4e check NaN for hypernetwork tuning 2022-10-15 17:15:26 +03:00