Compare commits

...

170 Commits

Author SHA1 Message Date
James Seibel d156772438 remove dev from the version number 2025-07-12 09:34:18 -05:00
James Seibel de7ae41769 Fix API config renderingEnabled() changing the user value
Fixes #1083
2025-07-12 08:16:35 -05:00
James Seibel 618ad1938b full data DTO close data source if corrupted 2025-07-10 22:24:33 -05:00
James Seibel 5b10263f82 minor format cleanup 2025-07-10 07:27:23 -05:00
James Seibel 34f914c52f Mark a unit test as deprecated
Done to suppress warnings in compiler log
2025-07-10 07:26:41 -05:00
James Seibel 67b766c674 Fix monoliths when connected to a server 2025-07-10 07:26:14 -05:00
James Seibel a3e7469203 Fixes !1078 (lag due to beacon updating on server) 2025-07-09 07:28:25 -05:00
James Seibel 4ecaa6a9a1 Potentially fix an issue with AMD GPU shader compiling
Fix from Cortex and the Canvas mod
2025-07-08 07:22:36 -05:00
James Seibel 24f9dadc58 Disable vanilla fading when shaders are active 2025-07-07 07:49:36 -05:00
s809 b3ebaffa85 Disable enableAdaptiveTransferSpeed bby default 2025-07-01 22:03:12 +05:00
James Seibel b7ac1909d6 Fix config UI changes not always saving 2025-07-01 07:45:26 -05:00
James Seibel 32c3118afa comment out Z_STD compression 2025-06-30 06:54:27 -05:00
James Seibel 3a525f53f0 Rename world gen mode "Internal Server" -> "Full - Save Chunks" 2025-06-28 13:57:58 -05:00
James Seibel f3947312c1 Re-Add Z_STD compression for testing 2025-06-28 11:37:06 -05:00
James Seibel 839ea1e778 increase ram amount for unit tests
needed for compression tests
2025-06-28 11:36:17 -05:00
James Seibel d2becd2c03 Fix rare fade error when restarting the LodRenderer 2025-06-28 10:45:36 -05:00
James Seibel 7d87347199 Fix multiplayer null pointer 2025-06-28 09:22:23 -05:00
James Seibel f4117751c9 Fix world-gen progress not showing in release builds 2025-06-27 07:29:31 -05:00
James Seibel a8a085f296 Move RenderState to core 2025-06-26 07:50:53 -05:00
James Seibel 317319593e rename renderDeferredLods -> renderDeferredLodsForShaders 2025-06-25 07:47:08 -05:00
James Seibel 4633f90a03 Add null handling to ServerPlayerStateManager.handlePluginMessage() 2025-06-25 07:45:43 -05:00
James Seibel 5802bbb3f3 keep cave culling for medium quality preset
This may be changed back at some point in the future, but depending on the usecase (IE vanilla survival) cave culling is generally better than not having it, and if people see weirdness they can probably guess that increasing the quality preset may fix it.
2025-06-23 07:23:26 -05:00
James Seibel e93d5b90f1 Disable cave culling for medium quality and higher 2025-06-17 07:15:28 -05:00
James Seibel 9be56607a5 Reduce stuttering with fast world gen 2025-06-14 16:17:28 -05:00
James Seibel 91743bf742 Add Api Before/After Text Create events
Deprecate DhApiColorDepthTextureCreatedEvent since it is less obvious when it fires
2025-06-09 07:50:21 -05:00
James Seibel d40d293f54 Fix hash collisions in FullDataPointIdMap 2025-06-06 07:43:38 -05:00
James Seibel a075e60e3e Fix GLMC.glDeleteTextures() calls 2025-06-04 07:07:39 -05:00
s809 d72c7c3695 Check LOD timestamps in file handler threads 2025-06-03 23:41:47 +05:00
Ran 309fa07664 Merge branch 'fix_max_y' into 'main'
Fix max Y validation

See merge request distant-horizons-team/distant-horizons-core!85
2025-05-18 00:32:51 +00:00
Stewart Borle 0a017567c4 Fix max Y validation 2025-05-18 00:32:51 +00:00
James Seibel e01261da5c Remove line ending from editorconfig
Done to fix some issues with some devs on linux
2025-05-17 11:47:00 -05:00
James Seibel a0879d07c5 json indent 2 -> 4
for consistency
2025-05-17 11:25:18 -05:00
Ran bbb15263f2 Fix gradle versioning 2025-05-03 11:21:05 +10:00
s809 5ca3563c66 Bump protocol version 2025-05-03 00:08:03 +05:00
s809 30256a2779 Send scaled generation bounds coordinates 2025-05-03 00:08:03 +05:00
Ran 4b4f10f5e6 Fix gradle versioning for core application 2025-05-02 12:44:25 +10:00
s809 ad995544f7 Use bytesReceived instead of decreasing multiplicatively 2025-04-20 23:59:34 +05:00
s809 d521e931f4 Change data send tick rate 4 -> 20 2025-04-20 18:26:07 +05:00
s809 dd30a8274a Add a config entry and refactor 2025-04-20 18:25:27 +05:00
s809 3ca5efadc9 Adaptive data transfer speed 2025-04-20 03:02:18 +05:00
Ran 09174c2d2a Improve LodDataBuilder.java
- Use bitwise modulo
- Don't compute certain things 256 times when they can be computed once.
- Removed expressions that are always false
- Improved comments
2025-04-11 11:24:16 +10:00
James Seibel e079b28e77 maybe break n-sized rendering but fix LOD loading getting stuck 2025-04-07 06:56:53 -05:00
James Seibel 136124a703 up version number 2.3.2 -> 2.3.3 2025-04-05 09:11:19 -05:00
James Seibel 3ed50e5134 remove dev from version number 2025-04-05 09:10:01 -05:00
James Seibel b5e3e6867c Improve DH world gen progress message 2025-04-02 07:25:14 -05:00
James Seibel 3e04342148 Add FIXME comments to Lod and Fade renderers 2025-04-02 07:24:38 -05:00
James Seibel 6699b568df Fix memory leaks due to un-closed thread pools and worlds
How did it take this long to realize the DhWorld objects were never being closed?
2025-03-30 17:30:57 -05:00
James Seibel 53bee4ad42 Remove unused code in LodRenderer 2025-03-30 16:55:01 -05:00
James Seibel 5d5e462221 Fix the sun/moon and stars not rendering 2025-03-30 16:49:58 -05:00
James Seibel d9b924cfed Fix beacon beams now going through some blocks 2025-03-30 15:23:19 -05:00
James Seibel 8bd70d593c Fix flashing on MC 1.21.5 in non-overworld dimensions 2025-03-30 14:36:51 -05:00
James Seibel 5597044604 don't log InterruptedException during threadPool shutdown 2025-03-29 20:11:31 -05:00
James Seibel 5d7c043d06 Fix fog for MC 1.16.5 2025-03-29 19:22:51 -05:00
James Seibel 4aac61b37f hide repo double close warnings in release 2025-03-29 15:39:45 -05:00
James Seibel 22460fa1f5 Fix duplicate world gen due to short memoization time
Reverts 276f2adf00
2025-03-29 15:30:28 -05:00
James Seibel 2d127c7d98 Fix an infinite loop in the lighting engine
Not sure how I didn't catch this until MC 1.21.5
2025-03-29 15:29:34 -05:00
James Seibel 91e17c420a Fix SSAO applying to sky 2025-03-29 10:31:48 -05:00
James Seibel 93f5a85cb5 Fix MC 1.21.5 rendering and bright glass on sky 2025-03-29 10:31:34 -05:00
James Seibel b275971486 re-add stencil to GL state
shouldn't be needed, but just in case
2025-03-29 09:52:41 -05:00
James Seibel 1234ff4d28 up version number 2.3.1 -> 2.3.2 2025-03-25 07:17:27 -05:00
James Seibel f9bd7e2daf remove dev from version number 2025-03-25 07:16:43 -05:00
James Seibel 8ec4e235eb Add config to only log GL errors once 2025-03-19 22:02:53 -05:00
James Seibel b8a59d0ef6 Attempt to fix Linux complaining about glIsFramebuffer() 2025-03-19 18:34:02 -05:00
James Seibel e500143781 Potentially fix GL errors when accessing the default FBO on Linux 2025-03-19 17:00:30 -05:00
James Seibel 406468b54c Fix restoring textures to the default FBO 2025-03-18 20:18:13 -05:00
James Seibel 6857300ae2 Add stack tracing to GL error logging 2025-03-18 18:10:00 -05:00
James Seibel 6775ee23c3 fix passing in the wrong flags to glBufferStorage()
Might Resolve #964 and #950
2025-03-18 07:43:20 -05:00
James Seibel 44645943e2 Fix terrain API repo failing if no cache was provided 2025-03-18 07:42:31 -05:00
James Seibel f385c4a56b Fix changing graphics settings on world load via API 2025-03-16 14:29:56 -05:00
James Seibel 0cf5e6d594 Fix GL error logging 2025-03-14 10:17:52 -05:00
James Seibel 7b5b8da0d2 decrease gen message timeout 5 -> 2 seconds
done to make it consistently appear
2025-03-13 21:17:01 -05:00
James Seibel 851f2ccd06 Add additional error checking/handling to Shader compiling 2025-03-13 21:12:29 -05:00
James Seibel 6c40389c07 up version number 2.3.0-b -> 2.3.1-b-dev 2025-03-08 08:11:01 -06:00
James Seibel fada9e4cf6 Fix repo leak unit test failing in release 2025-03-08 08:10:43 -06:00
James Seibel 06198fdbb8 Revert "temporarily disable sqlite tests for release"
This reverts commit ebc1114a51.
2025-03-06 07:43:14 -06:00
James Seibel 3158eed5a3 Merge branch 'main' of gitlab.com:distant-horizons-team/distant-horizons-core 2025-03-06 07:42:32 -06:00
James Seibel e701c0e5ea remove dev from version number 2025-03-06 07:40:49 -06:00
James Seibel ebc1114a51 temporarily disable sqlite tests for release 2025-03-06 07:40:24 -06:00
s809 d2ff4a5806 Add some debugging info for DTOs 2025-03-02 20:08:45 +05:00
s809 eb8563482e Replace chunk counts with speed in pregen 2025-02-27 21:08:27 +05:00
s809 b53c33e454 Make generation info text a bit clearer 2025-02-27 21:08:02 +05:00
s809 2483671e5e Should be division instead of multiplication 2025-02-26 23:16:58 +05:00
s809 cc4733b052 Offset generation bounds by teleportation scale 2025-02-26 22:13:38 +05:00
James Seibel 34e5463718 Merge branch 'main' of gitlab.com:distant-horizons-team/distant-horizons-core 2025-02-25 07:25:49 -06:00
James Seibel 53011a13be duct tape fix to reduce chance of LOD uploading requiring MC reboot 2025-02-25 07:25:46 -06:00
s809 1c579675a2 Remember split section responses temporarily 2025-02-22 20:55:10 +05:00
James Seibel 69a4e6b27e Add TODO about why LODs sometimes fail to load 2025-02-22 08:55:41 -06:00
s809 b05eb78f3a Fix foreground thread sometimes blocking server shutdown 2025-02-19 21:17:38 +05:00
s809 83fabe3ee8 Show section numbers in pregen 2025-02-19 20:37:26 +05:00
s809 fdfab2b3a8 Use another method for enforcing non nsized generation 2025-02-19 20:15:34 +05:00
James Seibel 45c67d057a Fix IDhApiConfigValue.clearValue() failing for some deprecated functions 2025-02-17 21:16:22 -06:00
James Seibel c296795280 Fix DB leaks in FullDataV2Repo 2025-02-16 20:07:00 -06:00
James Seibel 2deb24ec1e Add javadocs to ClientWrapper getPlayer Pos methods 2025-02-16 19:53:44 -06:00
James Seibel 5ab7a3030a Fix DB leaks in FullDataV2Repo 2025-02-16 19:52:48 -06:00
James Seibel 1af4d23c14 improve DB leak tracking exception handling 2025-02-16 19:52:37 -06:00
James Seibel 977204abf0 Add DB leak tracking 2025-02-16 19:34:13 -06:00
James Seibel 276f2adf00 Revert 10 minute memoization for world gen
I thought this was only an issue for N-sized generation, but in testing found it to still be an issue for max-detail retrieval as well.
This will have to be looked into more another time
2025-02-15 11:56:21 -06:00
James Seibel 1b3c9e1a89 Fix beacon culling with auto overdraw prevention 2025-02-15 11:12:46 -06:00
James Seibel 6fbe0a9e72 Add missing cave blocks for cave culling 2025-02-15 11:06:43 -06:00
James Seibel 11a2b8bf5b Add TODO to PriorityTaskPicker about VisualVM 2025-02-15 11:06:30 -06:00
James Seibel 99f2d2f844 Add TODO comment about Immersive Portals only rendering 1 level 2025-02-14 07:48:05 -06:00
s809 a5c029203c Invert generateOnlyInHighestDetail and rename to enableNSizedGeneration 2025-02-11 22:08:25 +05:00
James Seibel 84015e4a40 Put N-sized generation and upsampling behind experimental configs 2025-02-11 07:47:24 -06:00
James Seibel 08f63470a5 Fix auto updater failing for nightly builds 2025-02-10 07:46:43 -06:00
James Seibel f2404b6455 remove unneeded IVersionConstant methods 2025-02-08 11:39:18 -06:00
James Seibel f20231ccbc fix rare null pointer in sharedApi 2025-02-08 11:38:36 -06:00
s809 3a94bbe804 Reduce queue size back to improve responsiveness 2025-02-07 23:23:15 +05:00
s809 15f1754922 Improve ordering of reading positions to update a bit 2025-02-07 23:21:41 +05:00
s809 28448941e1 Keep update propagation queue filled 2025-02-07 23:20:24 +05:00
James Seibel 18c29b9810 Attempt to fix threadpool shutdown rejection exception 2025-02-07 07:25:57 -06:00
James Seibel fa66cefbe2 Add comments to LodRenderSection memoized gen positions
also increase timeout from 15 sec -> 10 minutes
 - done to test if memoization is actually needed
2025-02-07 07:14:50 -06:00
James Seibel f7dc46cb55 Increase full data update task count to reduce down time 2025-02-06 20:10:30 -06:00
James Seibel 5cebee3be4 Flush world gen memory cache when full 2025-02-06 20:08:28 -06:00
s809 532ac8fe01 Fix incorrect distance being used in update propagation SQL and reduce queue size 2025-02-07 01:05:14 +05:00
James Seibel 8385eeb62c Merge branch 'main' of gitlab.com:distant-horizons-team/distant-horizons-core 2025-02-05 17:31:26 -06:00
James Seibel 95db6885e7 add error logging to FullDataRequestHandler 2025-02-05 17:31:13 -06:00
James Seibel 10a3840373 Fix empty data sources when moving in multiplayer or with N-sized world gen
Increases Protocol version 9 -> 10
2025-02-05 17:30:59 -06:00
James Seibel cedaaa8a2e replace a few implicit datasource V2 repo statement setters 2025-02-04 19:29:39 -06:00
James Seibel 2c7f11c722 Improve DhApiBeforeRenderEvent javadocs 2025-02-03 20:30:28 -06:00
James Seibel 4fbda8f02b Fix render enabled config getting set by world gen progress config 2025-02-02 19:52:31 -06:00
James Seibel b0bd536248 Fix compiling with missing "E" 2025-02-02 15:52:17 -06:00
s809 a3ed0012e3 Balance tasks in thread pool using elapsed time instead of priorities 2025-02-02 20:30:35 +05:00
s809 9952481d77 Do not request already fulfilled sections again until some time passes 2025-02-02 20:30:35 +05:00
s809 5e137ee10d Auto-move old save data to new location 2025-02-02 20:30:35 +05:00
James Seibel f02ea68b6f Add missing Enum prefix to RequestResult -> ERequestResult 2025-02-01 16:08:15 -06:00
s809 1041e0a4dd Remove generationProgressDisableMessageDisplayTimeInSeconds from server config command 2025-02-01 19:38:43 +05:00
s809 6fb862ecfe Add GUI description for generateOnlyInHighestDetail 2025-02-01 19:38:25 +05:00
s809 1f8013c1cf Use generateOnlyInHighestDetail client-side 2025-02-01 18:30:07 +05:00
s809 157d72d8dc Decrease delay between missing generation rechecks 2025-01-31 14:54:18 +05:00
James Seibel 2c077f5224 Fix a null pointer in the chunk update queue 2025-01-30 20:13:53 -06:00
s809 6e5bd02ae0 Fix beacon beams flickering 2025-01-30 22:30:02 +05:00
s809 a7578b2a72 Process chunks only once with real-time updates enabled 2025-01-30 21:47:31 +05:00
s809 041cf4e0d4 Fix nightly self-updater after moving jars into zip root 2025-01-30 18:11:34 +05:00
s809 bb1154b036 Revert "Improve chunk processing throughput"
This reverts commit dd3903f66e.
2025-01-28 20:05:59 +05:00
James Seibel 9c9c90e786 Improve world gen import hiding message 2025-01-26 17:47:28 -06:00
James Seibel 3dbd05a4ae minor beacon beam height merge cleanup 2025-01-26 17:41:46 -06:00
James Seibel 042a0b6853 Merge branch 'distant-horizons-core-main' 2025-01-26 17:40:15 -06:00
James Seibel 39c621b8d9 Remove locks from LodRenderSection uploading 2025-01-26 17:12:39 -06:00
James Seibel dd3903f66e Improve chunk processing throughput 2025-01-26 17:05:37 -06:00
James Seibel 2d1859c77d change low memory warning to 4GB or more 2025-01-26 16:53:11 -06:00
James Seibel d62a801c43 Reduce locking in SharedApi.UpdateChunkPosManager 2025-01-25 10:10:10 -06:00
James Seibel cb40336fda Increase rolling average window for world gen queue
This should reduce fluctuations a bit
2025-01-24 21:53:13 -06:00
James Seibel 766c831af0 fix recalculate heightmap breaking stairs, slabs, and glass 2025-01-24 07:22:11 -06:00
s809 736df9f848 Check if session is ready before ignoring local chunks 2025-01-24 11:20:37 +05:00
James Seibel a347caafed Fix holes when moving with N-sized world gen/server side support 2025-01-23 19:44:58 -06:00
James Seibel 2d5902df28 Fix data source leaks for custom world generators 2025-01-23 19:38:01 -06:00
James Seibel 29e496757a Merge branch 'main' of gitlab.com:distant-horizons-team/distant-horizons-core 2025-01-23 19:16:33 -06:00
James Seibel 7cf05ed31d Fix rare concurrency error on world gen shutdown 2025-01-23 19:15:55 -06:00
James Seibel e7eb8e24ae Speed up PhantomArrayListPool for large checkouts 2025-01-23 19:15:42 -06:00
s809 cdca7723a7 Ignore local chunks if realtime updates are enabled 2025-01-23 23:21:13 +05:00
s809 e0a0ba5222 Fix full data source being released too early 2025-01-23 22:06:20 +05:00
James Seibel 0f88c7c231 Merge branch 'main' of gitlab.com:distant-horizons-team/distant-horizons-core 2025-01-22 21:31:20 -06:00
James Seibel d9911f64b9 Simplify Full data hash logic to speed up saving 2025-01-22 21:31:07 -06:00
s809 8bddd6d503 Fix column order check breaking on tiny columns 2 2025-01-23 00:18:33 +05:00
s809 9b261f6472 Fix column order check breaking on tiny columns 2025-01-23 00:15:12 +05:00
James Seibel 00559b5d34 Remove unneeded locks and speed up FullDataId Entry retrieval 2025-01-22 07:16:04 -06:00
James Seibel 9cae54a079 Show instructions to disable world gen progress message for short time 2025-01-21 07:49:30 -06:00
James Seibel 363ec76450 fix isClosedException name 2025-01-21 07:07:33 -06:00
s809 ebd00df388 Fix task splitting causing generation of already generated sections 2025-01-21 17:26:54 +05:00
James Seibel 13882f44ce minor LodRenderSection rename 2025-01-20 21:51:04 -06:00
James Seibel fce1fa3f41 Fix cached RenderSource closing while in use 2025-01-20 21:50:33 -06:00
James Seibel fab8191ddd remove unneeded wal flush logic 2025-01-20 07:39:04 -06:00
James Seibel 582541d240 handle additional DB closed message on DTO get 2025-01-20 07:38:40 -06:00
s809 f609dcb468 Merge remote-tracking branch 'origin/main' 2025-01-20 11:00:13 +05:00
s809 a69936ca69 Merge branch 'feature/generation-bounds' 2025-01-20 10:59:11 +05:00
James Seibel 8c81c867b6 merge 2025-01-19 17:42:45 -06:00
James Seibel 995f80d553 Fix beacons disappearing and not updating correctly 2025-01-19 17:42:14 -06:00
s809 08f36b4371 Lower the log level of rate limit hits 2025-01-18 17:02:35 +05:00
Jan Trummer 9dcc7e1ad2 Replace set with setMinDefaultMax 2025-01-11 16:05:58 +01:00
Jan Trummer 72139f1f59 Add config to set max beacon render height 2025-01-11 13:38:44 +01:00
128 changed files with 4612 additions and 2390 deletions
+1 -2
View File
@@ -4,7 +4,6 @@
[*]
charset = utf-8
end_of_line = crlf
indent_size = 4
indent_style = space
insert_final_newline = false
@@ -537,7 +536,7 @@ ij_groovy_wrap_chain_calls_after_dot = false
ij_groovy_wrap_long_lines = false
[{*.har,*.json,*.png.mcmeta,mcmod.info,pack.mcmeta}]
indent_size = 2
indent_size = 4
ij_json_array_wrapping = split_into_lines
ij_json_keep_blank_lines_in_code = 0
ij_json_keep_indents_on_empty_lines = false
@@ -39,8 +39,8 @@ public enum EDhApiDataCompressionMode
/**
* Should only be used internally and for unit testing. <br><br>
*
* Read Speed: 1.64 MS / DTO <br>
* Write Speed: 12.44 MS / DTO <br>
* Read Speed: 6.09 MS / DTO <br>
* Write Speed: 6.01 MS / DTO <br>
* Compression ratio: 1.0 <br>
*/
@DisallowSelectingViaConfigGui
@@ -49,28 +49,29 @@ public enum EDhApiDataCompressionMode
/**
* Extremely fast (often faster than uncompressed), but generally poor compression. <br><br>
*
* Read Speed: 1.85 MS / DTO <br>
* Write Speed: 9.46 MS / DTO <br>
* Compression ratio: 0.3638 <br>
* Read Speed: 3.25 MS / DTO <br>
* Write Speed: 5.99 MS / DTO <br>
* Compression ratio: 0.4513 <br>
*/
LZ4(1),
/*
* Decent speed and good compression. <br><br>
*
* Read Speed: 11.78 MS / DTO <br>
* Write Speed: 16.76 MS / DTO <br>
* Compression ratio: 0.2199 <br>
*/
//@Deprecated
///**
// * Decent speed and good compression. <br><br>
// *
// * Read Speed: 9.31 MS / DTO <br>
// * Write Speed: 15.13 MS / DTO <br>
// * Compression ratio: 0.2606 <br>
// */
////@DisallowSelectingViaConfigGui
//Z_STD(2),
/**
* Extremely slow, but very good compression. <br><br>
*
* Read Speed: 12.25 MS / DTO <br>
* Write Speed: 490.07 MS / DTO <br>
* Compression ratio: 0.1242 <br>
* Read Speed: 13.29 MS / DTO <br>
* Write Speed: 70.95 MS / DTO <br>
* Compression ratio: 0.2068 <br>
*/
LZMA2(3);
@@ -20,6 +20,8 @@
package com.seibel.distanthorizons.api.enums.worldGeneration;
/**
* DOWN_SAMPLED, <br>
*
* EMPTY, <br>
* STRUCTURE_START, <br>
* STRUCTURE_REFERENCE, <br>
@@ -37,6 +39,14 @@ package com.seibel.distanthorizons.api.enums.worldGeneration;
*/
public enum EDhApiWorldGenerationStep
{
/**
* Only used when using N-sized world generators or server-side retrieval.
* This denotes that the given datasource was created using lower quality LOD data from above it in the quad tree. <br>
*
* This isn't a valid option for queuing world generation.
*/
DOWN_SAMPLED(-1, "down_sampled"),
EMPTY(0, "empty"),
STRUCTURE_START(1, "structure_start"),
STRUCTURE_REFERENCE(2, "structure_reference"),
@@ -9,12 +9,15 @@ package com.seibel.distanthorizons.api.interfaces.data;
* @version 2024-7-14
* @since API 3.0.0
*/
public interface IDhApiTerrainDataCache
public interface IDhApiTerrainDataCache // TODO should this be AutoClosable?
{
/**
* Removes any data that's currently stored in this cache.
* This cane be done to free up memory or invalidate
* the cache so fresh data can be pulled in.
* <br><br>
* This should be called before de-referencing this object
* so DH can handle any necessary cleanup for internal objects.
*/
void clear();
@@ -0,0 +1,48 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.api.methods.events.abstractEvents;
import com.seibel.distanthorizons.api.methods.events.interfaces.IDhApiEvent;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiEventParam;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiTextureCreatedParam;
/**
* Called after Distant Horizons (re)creates
* the color and depth textures it renders to. <br>
*
* @author James Seibel
* @version 2025-6-9
* @since API 4.1.0
*/
public abstract class DhApiAfterColorDepthTextureCreatedEvent implements IDhApiEvent<DhApiTextureCreatedParam>
{
/** Fired before Distant Horizons creates. */
public abstract void onResize(DhApiEventParam<DhApiTextureCreatedParam> event);
//=========================//
// internal DH API methods //
//=========================//
@Override
public final void fireEvent(DhApiEventParam<DhApiTextureCreatedParam> event) { this.onResize(event); }
}
@@ -0,0 +1,49 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.api.methods.events.abstractEvents;
import com.seibel.distanthorizons.api.methods.events.interfaces.IDhApiEvent;
import com.seibel.distanthorizons.api.methods.events.interfaces.IDhApiEventParam;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiEventParam;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiTextureCreatedParam;
/**
* Called before Distant Horizons (re)creates
* the color and depth textures it renders to. <br>
*
* @author James Seibel
* @version 2025-6-9
* @since API 4.1.0
*/
public abstract class DhApiBeforeColorDepthTextureCreatedEvent implements IDhApiEvent<DhApiTextureCreatedParam>
{
/** Fired before Distant Horizons creates. */
public abstract void onResize(DhApiEventParam<DhApiTextureCreatedParam> event);
//=========================//
// internal DH API methods //
//=========================//
@Override
public final void fireEvent(DhApiEventParam<DhApiTextureCreatedParam> event) { this.onResize(event); }
}
@@ -25,11 +25,16 @@ import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhAp
/**
* Called before Distant Horizons starts rendering a frame. <br>
* Canceling the event will prevent DH from rendering that frame.
* Canceling the event will prevent DH from rendering that frame. <br> <br>
*
* This is called before DH starts modifying the GL state.
* If you want to inject into DH's rendering pass, use {@link DhApiBeforeRenderPassEvent} instead.
*
* @author James Seibel
* @version 2023-6-23
* @since API 1.0.0
*
* @see DhApiBeforeRenderPassEvent
*/
public abstract class DhApiBeforeRenderEvent implements IDhApiCancelableEvent<DhApiRenderParam>
{
@@ -22,15 +22,18 @@ package com.seibel.distanthorizons.api.methods.events.abstractEvents;
import com.seibel.distanthorizons.api.methods.events.interfaces.IDhApiEvent;
import com.seibel.distanthorizons.api.methods.events.interfaces.IDhApiEventParam;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiEventParam;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiTextureCreatedParam;
/**
* Called whenever Distant Horizons (re)creates
* Called before Distant Horizons (re)creates
* the color and depth textures it renders to. <br>
*
* @author James Seibel
* @version 2024-3-2
* @since API 2.0.0
* @deprecated Replaced by {@link DhApiBeforeColorDepthTextureCreatedEvent} since this event's name isn't obvious when it fires.
*/
@Deprecated
public abstract class DhApiColorDepthTextureCreatedEvent implements IDhApiEvent<DhApiColorDepthTextureCreatedEvent.EventParam>
{
/** Fired before Distant Horizons creates. */
@@ -73,6 +76,15 @@ public abstract class DhApiColorDepthTextureCreatedEvent implements IDhApiEvent<
this.newHeight = newHeight;
}
public EventParam(DhApiTextureCreatedParam textureCreatedParam)
{
this.previousWidth = textureCreatedParam.previousWidth;
this.previousHeight = textureCreatedParam.previousHeight;
this.newWidth = textureCreatedParam.newWidth;
this.newHeight = textureCreatedParam.newHeight;
}
@Override
@@ -0,0 +1,68 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.api.methods.events.sharedParameterObjects;
import com.seibel.distanthorizons.api.enums.rendering.EDhApiRenderPass;
import com.seibel.distanthorizons.api.methods.events.interfaces.IDhApiEventParam;
import com.seibel.distanthorizons.api.objects.math.DhApiMat4f;
/**
* Contains information relevant to when Distant Horizons (re)creates
* depth/color textures for rendering.
*
* @author James Seibel
* @version 2025-6-9
* @since API 4.1.0
*/
public class DhApiTextureCreatedParam implements IDhApiEventParam
{
/** Measured in pixels */
public final int previousWidth;
/** Measured in pixels */
public final int previousHeight;
/** Measured in pixels */
public final int newWidth;
/** Measured in pixels */
public final int newHeight;
public DhApiTextureCreatedParam(
int previousWidth, int previousHeight,
int newWidth, int newHeight)
{
this.previousWidth = previousWidth;
this.previousHeight = previousHeight;
this.newWidth = newWidth;
this.newHeight = newHeight;
}
@Override
public DhApiTextureCreatedParam copy()
{
return new DhApiTextureCreatedParam(
this.previousWidth, this.previousHeight,
this.newWidth, this.newHeight
);
}
}
@@ -87,7 +87,20 @@ public class DhApiConfigValue<coreType, apiType> implements IDhApiConfigValue<ap
}
}
public boolean clearValue() { return this.setValue(null); }
public boolean clearValue()
{
if (this.configEntry.getAllowApiOverride())
{
// no converter should be used here since null objects may need to be handled differently
// TODO the API should just have a bool to keep track of whether the API value is in use instead of using NULL
this.configEntry.setApiValue(null);
return true;
}
else
{
return false;
}
}
public boolean getCanBeOverrodeByApi() { return this.configEntry.getAllowApiOverride(); }
@@ -31,21 +31,21 @@ public final class ModInfo
public static final String DEDICATED_SERVER_INITIAL_PATH = "dedicated_server_initial";
/** Incremented every time any packets are added, changed or removed, with a few exceptions. */
public static final int PROTOCOL_VERSION = 9;
public static final int PROTOCOL_VERSION = 11;
public static final String WRAPPER_PACKET_PATH = "message";
/** The internal mod name */
public static final String NAME = "DistantHorizons";
/** Human-readable version of NAME */
public static final String READABLE_NAME = "Distant Horizons";
public static final String VERSION = "2.3.0-b-dev";
public static final String VERSION = "2.3.3-b";
/** Returns true if the current build is an unstable developer build, false otherwise. */
public static final boolean IS_DEV_BUILD = VERSION.toLowerCase().contains("dev");
/** This version should only be updated when breaking changes are introduced to the DH API */
public static final int API_MAJOR_VERSION = 4;
/** This version should be updated whenever new methods are added to the DH API */
public static final int API_MINOR_VERSION = 0;
public static final int API_MINOR_VERSION = 1;
/** This version should be updated whenever non-breaking fixes are added to the DH API */
public static final int API_PATCH_VERSION = 0;
@@ -31,14 +31,12 @@ import com.seibel.distanthorizons.coreapi.interfaces.config.IConverter;
public class RenderModeEnabledConverter implements IConverter<EDhApiRendererMode, Boolean>
{
@Override public EDhApiRendererMode convertToCoreType(Boolean renderingEnabled)
{
return renderingEnabled ? EDhApiRendererMode.DEFAULT : EDhApiRendererMode.DISABLED;
}
@Override
public EDhApiRendererMode convertToCoreType(Boolean renderingEnabled)
{ return renderingEnabled ? EDhApiRendererMode.DEFAULT : EDhApiRendererMode.DISABLED; }
@Override public Boolean convertToApiType(EDhApiRendererMode renderingMode)
{
return renderingMode == EDhApiRendererMode.DEFAULT;
}
@Override
public Boolean convertToApiType(EDhApiRendererMode renderingMode)
{ return renderingMode == EDhApiRendererMode.DEFAULT; }
}
+7 -1
View File
@@ -60,4 +60,10 @@ shadowJar {
def librariesLocation = "DistantHorizons.libraries"
// relocate "it.unimi.dsi.fastutil", "${librariesLocation}.unimi.dsi.fastutil"
mergeServiceFiles()
}
}
test {
// this is necessary specifically for the Compression tests since those
// need more than the default 512 MB of RAM
jvmArgs '-Xmx4096m'
}
@@ -26,6 +26,7 @@ import com.seibel.distanthorizons.api.interfaces.config.client.*;
import com.seibel.distanthorizons.api.objects.config.DhApiConfigValue;
import com.seibel.distanthorizons.api.enums.rendering.EDhApiRendererMode;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.coreapi.util.converters.RenderModeEnabledConverter;
public class DhApiGraphicsConfig implements IDhApiGraphicsConfig
{
@@ -60,7 +61,7 @@ public class DhApiGraphicsConfig implements IDhApiGraphicsConfig
@Override
public IDhApiConfigValue<Boolean> renderingEnabled()
{ return new DhApiConfigValue<Boolean, Boolean>(Config.Client.quickEnableRendering); }
{ return new DhApiConfigValue<EDhApiRendererMode, Boolean>(Config.Client.Advanced.Debugging.rendererMode, new RenderModeEnabledConverter()); }
@Override
public IDhApiConfigValue<EDhApiRendererMode> renderingMode()
@@ -197,10 +197,10 @@ public class DhApiTerrainDataRepo implements IDhApiTerrainDataRepo
ILevelWrapper coreLevelWrapper = (ILevelWrapper) levelWrapper;
if (!(apiDataCache instanceof DhApiTerrainDataCache))
// the data cache can be null, but must be our own implementation
if (apiDataCache != null
&& !(apiDataCache instanceof DhApiTerrainDataCache))
{
// custom level wrappers aren't supported,
// the API user must get a level wrapper from our code somewhere
return DhApiResult.createFail("Unsupported [" + IDhApiTerrainDataCache.class.getSimpleName() + "] implementation, only the core class [" + DhApiTerrainDataCache.class.getSimpleName() + "] is a valid parameter.");
}
DhApiTerrainDataCache dataCache = (DhApiTerrainDataCache) apiDataCache;
@@ -226,10 +226,9 @@ public class DhApiTerrainDataRepo implements IDhApiTerrainDataRepo
// get the data source //
//=====================//
FullDataSourceV2 dataSource = null;
try
{
FullDataSourceV2 dataSource = null;
// try using the cached data if possible
if (dataCache != null)
{
@@ -244,7 +243,12 @@ public class DhApiTerrainDataRepo implements IDhApiTerrainDataRepo
{
return DhApiResult.createFail("Unable to find/generate any data at the " + DhSectionPos.class.getSimpleName() + " [" + DhSectionPos.toString(sectionPos) + "].");
}
dataCache.add(sectionPos, dataSource);
// save to the cache if present
if (dataCache != null)
{
dataCache.add(sectionPos, dataSource);
}
}
@@ -316,6 +320,14 @@ public class DhApiTerrainDataRepo implements IDhApiTerrainDataRepo
LOGGER.error("Unexpected exception in getTerrainDataColumnArray. Error: [" + e.getMessage() + "]", e);
return DhApiResult.createFail("Unexpected exception: [" + e.getMessage() + "].");
}
finally
{
if (dataCache == null
&& dataSource != null)
{
dataSource.close();
}
}
}
@@ -89,6 +89,15 @@ public class ClientApi
/** this includes the is dev build message and low allocated memory warning */
private static final int MS_BETWEEN_STATIC_STARTUP_MESSAGES = 4_000;
/**
* This isn't the cleanest way of storing variables before passing them to the LOD renderer,
* but due to how mixins work and the inconsistency between MC versions,
* having a static object that stores a single frame's data
* is often the easiest solution. <br><br>
*
* Only downside is making sure each variable is populated before rendering.
*/
public static final RenderState RENDER_STATE = new RenderState();
private boolean isDevBuildMessagePrinted = false;
@@ -389,7 +398,7 @@ public class ClientApi
// rendering //
//===========//
/** Should be called before {@link ClientApi#renderDeferredLods} */
/** Should be called before {@link ClientApi#renderDeferredLodsForShaders} */
public void renderLods(IClientLevelWrapper levelWrapper, Mat4f mcModelViewMatrix, Mat4f mcProjectionMatrix, float partialTicks)
{ this.renderLodLayer(levelWrapper, mcModelViewMatrix, mcProjectionMatrix, partialTicks, false); }
@@ -397,9 +406,10 @@ public class ClientApi
* Only necessary when Shaders are in use.
* Should be called after {@link ClientApi#renderLods}
*/
public void renderDeferredLods(IClientLevelWrapper levelWrapper, Mat4f mcModelViewMatrix, Mat4f mcProjectionMatrix, float partialTicks)
public void renderDeferredLodsForShaders(IClientLevelWrapper levelWrapper, Mat4f mcModelViewMatrix, Mat4f mcProjectionMatrix, float partialTicks)
{ this.renderLodLayer(levelWrapper, mcModelViewMatrix, mcProjectionMatrix, partialTicks, true); }
private void renderLodLayer(
IClientLevelWrapper levelWrapper, Mat4f mcModelViewMatrix, Mat4f mcProjectionMatrix, float partialTicks,
boolean renderingDeferredLayer)
@@ -445,6 +455,25 @@ public class ClientApi
//Mat4f mcCombined = mcModelViewMatrix.copy();
//mcCombined.multiply(mcProjectionMatrix);
//
//com.seibel.distanthorizons.api.objects.math.DhApiMat4f dhCombined = renderEventParam.dhModelViewMatrix.copy();
//dhCombined.multiply(renderEventParam.dhProjectionMatrix);
//
//LOGGER.info("\n\n" +
// "API\n" +
// "Mc MVM: \n" + mcModelViewMatrix.toString() + "\n" +
// "Mc Proj: \n" + mcProjectionMatrix + "\n" +
// "Mc Combined:\n" + mcCombined.toString() + "\n" +
// "\n" +
// "DH MVM: \n" + renderEventParam.dhModelViewMatrix.toString() + "\n" +
// "DH Proj: \n" + renderEventParam.dhProjectionMatrix + "\n" +
// "DH Combined:\n" + mcCombined.toString()
//);
// render validation //
try
@@ -555,25 +584,26 @@ public class ClientApi
public void renderFadeOpaque(Mat4f mcModelViewMatrix, Mat4f mcProjectionMatrix, float partialTicks, IClientLevelWrapper level)
{
// only fade when DH is rendering
if (Config.Client.Advanced.Debugging.rendererMode.get() == EDhApiRendererMode.DEFAULT)
if (Config.Client.Advanced.Debugging.rendererMode.get() == EDhApiRendererMode.DEFAULT
// only fade when requested
&& Config.Client.Advanced.Graphics.Quality.vanillaFadeMode.get() == EDhApiMcRenderingFadeMode.DOUBLE_PASS
// don't fade when Iris shaders are active, otherwise the rendering can get weird
&& !DhApiRenderProxy.INSTANCE.getDeferTransparentRendering())
{
if (Config.Client.Advanced.Graphics.Quality.vanillaFadeMode.get() == EDhApiMcRenderingFadeMode.DOUBLE_PASS)
{
FadeRenderer.INSTANCE.render(mcModelViewMatrix, mcProjectionMatrix, partialTicks, level);
}
FadeRenderer.INSTANCE.render(mcModelViewMatrix, mcProjectionMatrix, partialTicks, level);
}
}
/** should be called after DH and MC finish rendering so we can smooth the transition between the two */
public void renderFade(Mat4f mcModelViewMatrix, Mat4f mcProjectionMatrix, float partialTicks, IClientLevelWrapper level)
{
// only fade when DH is rendering
if (Config.Client.Advanced.Debugging.rendererMode.get() == EDhApiRendererMode.DEFAULT)
if (Config.Client.Advanced.Debugging.rendererMode.get() == EDhApiRendererMode.DEFAULT
// only fade when requested
&& Config.Client.Advanced.Graphics.Quality.vanillaFadeMode.get() != EDhApiMcRenderingFadeMode.NONE
// don't fade when Iris shaders are active, otherwise the rendering can get weird
&& !DhApiRenderProxy.INSTANCE.getDeferTransparentRendering())
{
// fade if any level fading is active
if (Config.Client.Advanced.Graphics.Quality.vanillaFadeMode.get() != EDhApiMcRenderingFadeMode.NONE)
{
FadeRenderer.INSTANCE.render(mcModelViewMatrix, mcProjectionMatrix, partialTicks, level);
}
FadeRenderer.INSTANCE.render(mcModelViewMatrix, mcProjectionMatrix, partialTicks, level);
}
}
@@ -652,7 +682,8 @@ public class ClientApi
private void detectAndSendBootTimeWarnings()
{
// dev build
if (ModInfo.IS_DEV_BUILD && !this.isDevBuildMessagePrinted && MC_CLIENT.playerExists())
if (ModInfo.IS_DEV_BUILD
&& !this.isDevBuildMessagePrinted && MC_CLIENT.playerExists())
{
this.isDevBuildMessagePrinted = true;
this.lastStaticWarningMessageSentMsTime = System.currentTimeMillis();
@@ -669,7 +700,8 @@ public class ClientApi
// memory
if (this.staticStartupMessageSentRecently()) return;
if (!this.lowMemoryWarningPrinted && Config.Common.Logging.Warning.showLowMemoryWarningOnStartup.get())
if (!this.lowMemoryWarningPrinted
&& Config.Common.Logging.Warning.showLowMemoryWarningOnStartup.get())
{
this.lowMemoryWarningPrinted = true;
this.lastStaticWarningMessageSentMsTime = System.currentTimeMillis();
@@ -685,7 +717,7 @@ public class ClientApi
// orange text
"\u00A76" + "Distant Horizons: Low memory detected." + "\u00A7r \n" +
"Stuttering or low FPS may occur. \n" +
"Please increase Minecraft's available memory to 4 gigabytes. \n" +
"Please increase Minecraft's available memory to 4 GB or more. \n" +
"This warning can be disabled in DH's config under Advanced -> Logging. \n";
MC_CLIENT.sendChatMessage(message);
}
@@ -694,7 +726,8 @@ public class ClientApi
// high vanilla render distance
if (this.staticStartupMessageSentRecently()) return;
if (!this.highVanillaRenderDistanceWarningPrinted && Config.Common.Logging.Warning.showHighVanillaRenderDistanceWarning.get())
if (!this.highVanillaRenderDistanceWarningPrinted
&& Config.Common.Logging.Warning.showHighVanillaRenderDistanceWarning.get())
{
// DH generally doesn't need a vanilla render distance above 12
if (MC_RENDER.getRenderDistance() > 12)
@@ -721,7 +754,8 @@ public class ClientApi
{
if (this.lastStaticWarningMessageSentMsTime == 0)
{
return true;
// no static message has ever been sent
return false;
}
long timeSinceLastMessage = System.currentTimeMillis() - this.lastStaticWarningMessageSentMsTime;
@@ -740,4 +774,47 @@ public class ClientApi
*/
public void showOverlayMessageNextFrame(String message) { this.overlayMessageQueueForNextFrame.add(message); }
//================//
// helper classes //
//================//
public static class RenderState
{
public Mat4f mcModelViewMatrix = null;
public Mat4f mcProjectionMatrix = null;
public float frameTime = -1;
public void canRenderOrThrow() throws IllegalStateException
{
String errorReasons = "";
if (this.mcModelViewMatrix == null)
{
errorReasons += "no MVM Matrix, ";
}
if (this.mcProjectionMatrix == null)
{
errorReasons += "no Projection Matrix, ";
}
if (this.frameTime == -1)
{
errorReasons += "no Frame Time, ";
}
if (!errorReasons.isEmpty())
{
throw new IllegalStateException(errorReasons);
}
}
}
}
@@ -25,6 +25,7 @@ import com.seibel.distanthorizons.core.Initializer;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.generation.DhLightingEngine;
import com.seibel.distanthorizons.core.level.DhClientLevel;
import com.seibel.distanthorizons.core.level.IDhLevel;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.logging.f3.F3Screen;
@@ -49,7 +50,6 @@ import org.jetbrains.annotations.Nullable;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.locks.ReentrantLock;
/** Contains code and variables used by both {@link ClientApi} and {@link ServerApi} */
public class SharedApi
@@ -99,6 +99,11 @@ public class SharedApi
public static void setDhWorld(AbstractDhWorld newWorld)
{
AbstractDhWorld oldWorld = currentWorld;
if (oldWorld != null)
{
oldWorld.close();
}
currentWorld = newWorld;
// starting and stopping the DataRenderTransformer is necessary to prevent attempting to
@@ -231,6 +236,14 @@ public class SharedApi
return;
}
if (dhLevel instanceof DhClientLevel)
{
if (!((DhClientLevel) dhLevel).shouldProcessChunkUpdate(chunkWrapper.getChunkPos()))
{
return;
}
}
// shoudln't normally happen, but just in case
if (UPDATE_POS_MANAGER.contains(chunkWrapper.getChunkPos()))
{
@@ -491,11 +504,9 @@ public class SharedApi
/** keeps track of which chunks need to be updated */
private static class UpdateChunkPosManager
{
private final PriorityQueue<DhChunkPos> closestQueue;
private final PriorityQueue<DhChunkPos> furthestQueue;
private final HashMap<DhChunkPos, UpdateChunkData> updateDataByChunkPos;
private final ReentrantLock lock = new ReentrantLock();
private final PriorityBlockingQueue<DhChunkPos> closestQueue;
private final PriorityBlockingQueue<DhChunkPos> furthestQueue;
private final ConcurrentHashMap<DhChunkPos, UpdateChunkData> updateDataByChunkPos;
private DhChunkPos center;
private int maxSize = 500;
@@ -508,9 +519,9 @@ public class SharedApi
public UpdateChunkPosManager()
{
this.closestQueue = new PriorityQueue<>(Comparator.comparingDouble(pos -> pos.squaredDistance(this.center)));
this.furthestQueue = new PriorityQueue<>(Comparator.comparingDouble(pos -> ((DhChunkPos)pos).squaredDistance(this.center)).reversed());
this.updateDataByChunkPos = new HashMap<>();
this.closestQueue = new PriorityBlockingQueue<>(500, Comparator.comparingDouble(pos -> pos.squaredDistance(this.center)));
this.furthestQueue = new PriorityBlockingQueue<>(500, Comparator.comparingDouble(pos -> ((DhChunkPos)pos).squaredDistance(this.center)).reversed());
this.updateDataByChunkPos = new ConcurrentHashMap<>();
// defaulting to 0,0 is fine since it'll be updated once we start adding items
this.center = new DhChunkPos(0, 0);
}
@@ -521,50 +532,20 @@ public class SharedApi
// list/set methods //
//==================//
public boolean contains(DhChunkPos pos)
{
try
{
this.lock.lock();
return this.updateDataByChunkPos.containsKey(pos);
}
finally
{
this.lock.unlock();
}
}
public boolean contains(DhChunkPos pos) { return this.updateDataByChunkPos.containsKey(pos); }
public void clear()
{
try
{
this.lock.lock();
this.updateDataByChunkPos.clear();
this.closestQueue.clear();
this.furthestQueue.clear();
}
finally
{
this.lock.unlock();
}
this.updateDataByChunkPos.clear();
this.closestQueue.clear();
this.furthestQueue.clear();
}
public void removeItem(DhChunkPos pos)
{
try
{
this.lock.lock();
this.updateDataByChunkPos.remove(pos);
this.closestQueue.remove(pos);
this.furthestQueue.remove(pos);
}
finally
{
this.lock.unlock();
}
this.updateDataByChunkPos.remove(pos);
this.closestQueue.remove(pos);
this.furthestQueue.remove(pos);
}
/**
@@ -575,35 +556,29 @@ public class SharedApi
*/
public int addItem(DhChunkPos pos, UpdateChunkData updateData)
{
try
int remainingSlots = this.maxSize - this.updateDataByChunkPos.size();
if (this.updateDataByChunkPos.containsKey(pos))
{
this.lock.lock();
int remainingSlots = this.maxSize - this.updateDataByChunkPos.size();
if (this.updateDataByChunkPos.containsKey(pos))
// Chunk is already present in queue, no need to insert
return remainingSlots;
}
// If no slots are left, get one by removing the item furthest from the center
if (remainingSlots <= 0)
{
DhChunkPos furthest = this.furthestQueue.poll();
if (furthest != null)
{
// Chunk is already present in queue, no need to insert
return remainingSlots;
}
// If no slots are left, get one by removing the item furthest from the center
if (remainingSlots <= 0)
{
DhChunkPos furthest = this.furthestQueue.poll();
this.closestQueue.remove(furthest);
this.updateDataByChunkPos.remove(furthest);
}
this.updateDataByChunkPos.put(pos, updateData);
this.closestQueue.add(pos);
this.furthestQueue.add(pos);
return remainingSlots;
}
finally
{
this.lock.unlock();
}
this.updateDataByChunkPos.put(pos, updateData);
this.closestQueue.add(pos);
this.furthestQueue.add(pos);
return remainingSlots;
}
@@ -622,46 +597,33 @@ public class SharedApi
return;
}
try
this.center = newCenter;
// rebuild the priority queues to match the new center
this.closestQueue.clear();
this.furthestQueue.clear();
for (DhChunkPos pos : this.updateDataByChunkPos.keySet())
{
this.lock.lock();
this.center = newCenter;
// rebuild the priority queues to match the new center
this.closestQueue.clear();
this.furthestQueue.clear();
for (DhChunkPos pos : this.updateDataByChunkPos.keySet())
{
this.closestQueue.add(pos);
this.furthestQueue.add(pos);
}
}
finally
{
this.lock.unlock();
this.closestQueue.add(pos);
this.furthestQueue.add(pos);
}
}
public UpdateChunkData popClosest()
{
try
if (this.closestQueue.isEmpty())
{
this.lock.lock();
if (this.closestQueue.isEmpty())
{
return null;
}
DhChunkPos closest = this.closestQueue.poll();
this.furthestQueue.remove(closest);
return this.updateDataByChunkPos.remove(closest);
return null;
}
finally
DhChunkPos closest = this.closestQueue.poll();
if (closest == null)
{
this.lock.unlock();
return null;
}
this.furthestQueue.remove(closest);
return this.updateDataByChunkPos.remove(closest);
}
}
@@ -427,6 +427,15 @@ public class Config
+ "")
.build();
public static ConfigEntry<Integer> beaconRenderHeight = new ConfigEntry.Builder<Integer>()
.setMinDefaultMax(1, 6000, 6_000_000)
.comment(""
+ "Sets the maximum height at which beacons will render."
+ "This will only affect new beacons coming into LOD render distance."
+ "Beacons currently visible in LOD chunks will not be affected."
+ "")
.build();
public static ConfigEntry<Boolean> enableCloudRendering = new ConfigEntry.Builder<Boolean>()
.set(true)
.comment(""
@@ -699,16 +708,15 @@ public class Config
public static ConfigEntry<Boolean> enableCaveCulling = new ConfigEntry.Builder<Boolean>()
.set(true)
.setPerformance(EConfigEntryPerformance.HIGH)
.comment(""
+ "If enabled caves will be culled \n"
+ "If enabled caves won't be rendered. \n"
+ "\n"
+ "NOTE: This feature is under development and \n"
+ " it is VERY experimental! Please don't report \n"
+ " any issues related to this feature. \n"
+ "\n"
+ "Additional Info: Currently this cull all faces \n"
+ " with skylight value of 0 in dimensions that \n"
+ " does not have a ceiling.")
+ " Note: for some world types this can cause \n"
+ " overhangs or walls for floating objects. \n"
+ " Tweaking the caveCullingHeight, can resolve some \n"
+ " of those issues. \n"
+ "")
.addListener(ReloadLodsConfigEventHandler.INSTANCE)
.build();
@@ -753,20 +761,26 @@ public class Config
+ "Disable this if shadows render incorrectly.")
.build();
public static ConfigEntry<String> ignoredRenderBlockCsv = new ConfigEntry.Builder<String>()
public static ConfigEntry<String> ignoredRenderBlockCsv = new ConfigEntry.Builder<String>() // TODO accept wildcards
.set("minecraft:barrier,minecraft:structure_void,minecraft:light,minecraft:tripwire,minecraft:brown_mushroom")
.setAppearance(EConfigEntryAppearance.ONLY_IN_FILE) // only shown in file since the UI has a character limit
.comment(""
+ "A comma separated list of block resource locations that won't be rendered by DH. \n"
+ "Note: air is always included in this list. \n"
+ "Air is always included in this list. \n"
+ "Requires a restart to change. \n"
+ "")
.build();
public static ConfigEntry<String> ignoredRenderCaveBlockCsv = new ConfigEntry.Builder<String>()
.set("minecraft:glow_lichen,minecraft:rail,minecraft:water,minecraft:lava,minecraft:bubble_column")
public static ConfigEntry<String> ignoredRenderCaveBlockCsv = new ConfigEntry.Builder<String>() // TODO accept wildcards
.set("minecraft:glow_lichen,minecraft:rail,minecraft:water,minecraft:lava,minecraft:bubble_column," +
"minecraft:cave_vines_plant,minecraft:vine,minecraft:cave_vines,minecraft:short_grass,minecraft:tall_grass," +
"minecraft:small_dripleaf,minecraft:big_dripleaf,minecraft:big_dripleaf_stem,minecraft:sculk_vein")
.setAppearance(EConfigEntryAppearance.ONLY_IN_FILE) // only shown in file since the UI has a character limit
.comment(""
+ "A comma separated list of block resource locations that shouldn't be rendered \n"
+ "if they are in a 0 sky light underground area. \n"
+ "Note: air is always included in this list. \n"
+ "Air is always included in this list. \n"
+ "Requires a restart to change. \n"
+ "")
.build();
@@ -1001,9 +1015,23 @@ public class Config
public static class OpenGl
{
public static ConfigEntry<Boolean> overrideVanillaGLLogger = new ConfigEntry.Builder<Boolean>()
.set(ModInfo.IS_DEV_BUILD)
.set(true)
.comment(""
+ "Requires a reboot to change. \n"
+ "Defines how OpenGL errors are handled. \n "
+ "Requires rebooting Minecraft to change. \n"
+ "Will catch OpenGL errors thrown by other mods. \n"
+ "")
.build();
public static ConfigEntry<Boolean> onlyLogGlErrorsOnce = new ConfigEntry.Builder<Boolean>()
.set(true)
.comment(""
+ "If true each Open GL error will only be logged once. \n"
+ "Enabling this may cause some error logs to be missed. \n"
+ "Does nothing if overrideVanillaGLLogger is set to false. \n"
+ " \n"
+ "Generally this can be kept as 'true' to prevent log spam. \n"
+ "However, Please set this to 'false' if a developer needs your log to debug a GL issue. \n"
+ "")
.build();
@@ -1267,12 +1295,20 @@ public class Config
public static ConfigEntry<Integer> generationProgressDisplayIntervalInSeconds = new ConfigEntry.Builder<Integer>()
.setChatCommandName("generation.logInterval")
.setMinDefaultMax(1, 5, 60 * 60 * 4) // max = 4 hours
.setMinDefaultMax(1, 2, 60 * 60 * 4) // max = 4 hours
.comment(""
+ "How often should the distant generator progress be displayed? \n"
+ "")
.build();
public static ConfigEntry<Integer> generationProgressDisableMessageDisplayTimeInSeconds = new ConfigEntry.Builder<Integer>()
.setMinDefaultMax(0, 20, 60 * 60) // max = 1 hour
.comment(""
+ "For how many seconds should instructions for disabling the distant generator progress be displayed? \n"
+ "Setting this to 0 hides the instructional message so the world gen progress is shown immediately when it starts. \n"
+ "")
.build();
}
public static class LodBuilding
@@ -1306,20 +1342,20 @@ public class Config
+ EDhApiDataCompressionMode.UNCOMPRESSED + " \n"
+ "Should only be used for testing, is worse in every way vs ["+EDhApiDataCompressionMode.LZ4+"].\n"
+ "Expected Compression Ratio: 1.0\n"
+ "Estimated average DTO read speed: 1.64 milliseconds\n"
+ "Estimated average DTO write speed: 12.44 milliseconds\n"
+ "Estimated average DTO read speed: 3.25 milliseconds\n"
+ "Estimated average DTO write speed: 5.99 milliseconds\n"
+ "\n"
+ EDhApiDataCompressionMode.LZ4 + " \n"
+ "A good option if you're CPU limited and have plenty of hard drive space.\n"
+ "Expected Compression Ratio: 0.36\n"
+ "Expected Compression Ratio: 0.26\n"
+ "Estimated average DTO read speed: 1.85 ms\n"
+ "Estimated average DTO write speed: 9.46 ms\n"
+ "\n"
+ EDhApiDataCompressionMode.LZMA2 + " \n"
+ "Slow but very good compression.\n"
+ "Expected Compression Ratio: 0.14\n"
+ "Estimated average DTO read speed: 11.89 ms\n"
+ "Estimated average DTO write speed: 192.01 ms\n"
+ "Expected Compression Ratio: 0.2\n"
+ "Estimated average DTO read speed: 13.29 ms\n"
+ "Estimated average DTO write speed: 70.95 ms\n"
+ "")
.build();
@@ -1386,6 +1422,30 @@ public class Config
+ "")
.build();
public static ConfigCategory experimental = new ConfigCategory.Builder().set(Experimental.class).build();
public static class Experimental
{
public static ConfigEntry<Boolean> upsampleLowerDetailLodsToFillHoles = new ConfigEntry.Builder<Boolean>()
.set(false)
.comment(""
+ "When active DH will attempt to fill missing LOD data \n"
+ "with any data that is present in the tree, preventing holes when moving \n"
+ "when a N-sized generator (or server) is active. \n"
+ "\n"
+ "This is only used when N-sized world generation is available \n"
+ "and/or when on a server where [generateOnlyInHighestDetail] is false. \n"
+ "\n"
+ "Experimental:\n"
+ "Enabling this option will increase CPU and harddrive use\n"
+ "and may cause rendering bugs.\n"
+ "\n"
+ "")
.build();
}
}
public static class MultiThreading
@@ -1569,14 +1629,6 @@ public class Config
.setPerformance(EConfigEntryPerformance.HIGH)
.build();
public static ConfigEntry<Boolean> generateOnlyInHighestDetail = new ConfigEntry.Builder<Boolean>()
.setChatCommandName("generation.highestDetailOnly")
.set(false)
.comment(""
+ "Makes the server reject all generation requests for detail levels below the highest one.\n"
+ "")
.build();
public static ConfigEntry<Integer> generationBoundsX = new ConfigEntry.Builder<Integer>()
.setChatCommandName("generation.bounds.x")
.setAppearance(EConfigEntryAppearance.ONLY_IN_FILE)
@@ -1661,6 +1713,32 @@ public class Config
+ "Value of 0 disables the limit."
+ "")
.build();
public static ConfigEntry<Boolean> enableAdaptiveTransferSpeed = new ConfigEntry.Builder<Boolean>()
.set(false)
.comment(""
+ "Enables adaptive transfer speed based on client performance.\n"
+ "If true, DH will automatically adjust transfer rate to minimize connection lag.\n"
+ "If false, transfer speed will remain fixed.\n"
+ "")
.build();
public static ConfigCategory experimental = new ConfigCategory.Builder().set(Experimental.class).build();
public static class Experimental
{
public static ConfigEntry<Boolean> enableNSizedGeneration = new ConfigEntry.Builder<Boolean>()
.setChatCommandName("generation.nSized")
.set(false)
.comment(""
+ "When enabled on the client, this allows loading lower detail levels as needed to speed up terrain generation.\n"
+ "This must also be enabled on the server; otherwise, it will have no effect.\n"
+ "For better performance when switching LOD detail levels, enabling [upsampleLowerDetailLodsToFillHoles] is recommended.\n"
+ "")
.build();
}
}
@@ -35,8 +35,17 @@ public class QuickRenderToggleConfigEventHandler
/** private since we only ever need one handler at a time */
private QuickRenderToggleConfigEventHandler()
{
this.quickRenderChangeListener = new ConfigChangeListener<>(Config.Client.quickEnableRendering, (val) -> { Config.Client.Advanced.Debugging.rendererMode.set(Config.Client.quickEnableRendering.get() ? EDhApiRendererMode.DEFAULT : EDhApiRendererMode.DISABLED); });
this.rendererModeChangeListener = new ConfigChangeListener<>(Config.Client.Advanced.Debugging.rendererMode, (val) -> { Config.Client.quickEnableRendering.set(Config.Client.Advanced.Debugging.rendererMode.get() != EDhApiRendererMode.DISABLED); });
this.quickRenderChangeListener = new ConfigChangeListener<>(Config.Client.quickEnableRendering,
(val) -> {
Config.Client.Advanced.Debugging.rendererMode.set(Config.Client.quickEnableRendering.get()
? EDhApiRendererMode.DEFAULT
: EDhApiRendererMode.DISABLED);
});
this.rendererModeChangeListener = new ConfigChangeListener<>(Config.Client.Advanced.Debugging.rendererMode,
(val) -> {
Config.Client.quickEnableRendering.set(
Config.Client.Advanced.Debugging.rendererMode.get() != EDhApiRendererMode.DISABLED);
});
}
/**
@@ -45,7 +54,8 @@ public class QuickRenderToggleConfigEventHandler
*/
public void setUiOnlyConfigValues()
{
Config.Client.quickEnableRendering.set(Config.Client.Advanced.Debugging.rendererMode.get() != EDhApiRendererMode.DISABLED);
boolean enableRendering = Config.Client.Advanced.Debugging.rendererMode.get() != EDhApiRendererMode.DISABLED;
Config.Client.quickEnableRendering.set(enableRendering);
}
}
@@ -56,7 +56,8 @@ public class QuickShowWorldGenProgressConfigEventHandler
*/
public void setUiOnlyConfigValues()
{
Config.Client.quickEnableRendering.set(Config.Common.WorldGenerator.showGenerationProgress.get() != EDhApiDistantGeneratorProgressDisplayLocation.DISABLED);
boolean showProgress = Config.Common.WorldGenerator.showGenerationProgress.get() != EDhApiDistantGeneratorProgressDisplayLocation.DISABLED;
Config.Client.quickShowWorldGenProgress.set(showProgress);
}
}
@@ -105,6 +105,15 @@ public class RenderQualityPresetConfigEventHandler extends AbstractPresetConfigE
this.put(EDhApiQualityPreset.HIGH, true);
this.put(EDhApiQualityPreset.EXTREME, true);
}});
private final ConfigEntryWithPresetOptions<EDhApiQualityPreset, Boolean> caveCulling = new ConfigEntryWithPresetOptions<>(Config.Client.Advanced.Graphics.Culling.enableCaveCulling,
new HashMap<EDhApiQualityPreset, Boolean>()
{{
this.put(EDhApiQualityPreset.MINIMUM, true);
this.put(EDhApiQualityPreset.LOW, true);
this.put(EDhApiQualityPreset.MEDIUM, true);
this.put(EDhApiQualityPreset.HIGH, false);
this.put(EDhApiQualityPreset.EXTREME, false);
}});
@@ -123,6 +132,7 @@ public class RenderQualityPresetConfigEventHandler extends AbstractPresetConfigE
this.configList.add(this.ssaoEnabled);
this.configList.add(this.vanillaFade);
this.configList.add(this.dhDither);
this.configList.add(this.caveCulling);
for (ConfigEntryWithPresetOptions<EDhApiQualityPreset, ?> config : this.configList)
@@ -70,7 +70,12 @@ public class ConfigFileHandling
this.configBase = configBase;
this.configPath = configPath;
this.nightConfig = CommentedFileConfig.builder(this.configPath.toFile()).build();
this.nightConfig = CommentedFileConfig
.builder(this.configPath.toFile())
// sync is needed so file reading/writing only happens during locked sections,
// otherwise some GUI changes may be lost when changing screens
.sync()
.build();
}
@@ -321,10 +326,7 @@ public class ConfigFileHandling
*
* @apiNote This overwrites any value currently stored in the config
*/
public void loadNightConfig()
{
loadNightConfig(this.nightConfig);
}
public void loadNightConfig() { this.loadNightConfig(this.nightConfig); }
/**
* Does {@link CommentedFileConfig#load()} but with error checking
*
@@ -353,7 +355,7 @@ public class ConfigFileHandling
{
System.out.println("Creating file failed");
this.logger.error(ex);
SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class).crashMinecraft("Loading file and resetting config file failed at path [" + configPath + "]. Please check the file is ok and you have the permissions", ex);
SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class).crashMinecraft("Loading file and resetting config file failed at path [" + this.configPath + "]. Please check the file is ok and you have the permissions", ex);
}
}
@@ -26,6 +26,7 @@ import com.seibel.distanthorizons.core.config.listeners.IConfigListener;
import com.seibel.distanthorizons.core.config.types.enums.EConfigEntryAppearance;
import com.seibel.distanthorizons.core.config.types.enums.EConfigEntryPerformance;
import com.seibel.distanthorizons.coreapi.interfaces.config.IConfigEntry;
import org.jetbrains.annotations.Nullable;
import java.util.ArrayList;
import java.util.Arrays;
@@ -54,6 +55,8 @@ public class ConfigEntry<T> extends AbstractConfigType<T, ConfigEntry<T>> implem
* and any get() method calls will return the apiValue if it is set.
*/
public final boolean allowApiOverride;
/** Will be null if un-set */
@Nullable
private T apiValue;
@@ -132,9 +135,9 @@ public class ConfigEntry<T> extends AbstractConfigType<T, ConfigEntry<T>> implem
@Override
public T get()
{
if (allowApiOverride && apiValue != null)
if (this.allowApiOverride && this.apiValue != null)
{
return apiValue;
return this.apiValue;
}
return super.get();
@@ -29,13 +29,12 @@ import com.seibel.distanthorizons.core.wrapperInterfaces.block.IBlockStateWrappe
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IBiomeWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.IWrapperFactory;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.ILevelWrapper;
import it.unimi.dsi.fastutil.ints.Int2ReferenceOpenHashMap;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import java.io.*;
import java.util.*;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.concurrent.ConcurrentHashMap;
/**
* WARNING: This is not THREAD-SAFE! <br><br>
@@ -46,7 +45,7 @@ import java.util.concurrent.locks.ReentrantReadWriteLock;
* since it stringifies every block and biome name, which is quite bulky.
* It might be worth while to have a biome and block ID that then both get mapped
* to the data point ID to reduce file size.
* And/or it would be good to dynamically remove IDs that aren't currently in use.
* And/or it would be good to dynamically remove IDs that aren't currently in use.
*
* @author Leetom
*/
@@ -63,15 +62,12 @@ public class FullDataPointIdMap
private static final String BLOCK_STATE_SEPARATOR_STRING = "_DH-BSW_";
/** used when the data point map is running normally */
private final ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
/** should only be used for debugging */
private long pos;
/** The index should be the same as the Entry's ID */
private final ArrayList<Entry> entryList = new ArrayList<>();
private final HashMap<Entry, Integer> idMap = new HashMap<>();
private final ConcurrentHashMap<Entry, Integer> idMap = new ConcurrentHashMap<>();
private int cachedHashCode = 0;
@@ -89,34 +85,25 @@ public class FullDataPointIdMap
// getters //
//=========//
/** @throws IndexOutOfBoundsException if the given ID isn't in the {@link FullDataPointIdMap#entryList} */
private Entry getEntry(int id) throws IndexOutOfBoundsException
{
try
{
this.readWriteLock.readLock().lock();
Entry entry;
try
{
entry = this.entryList.get(id);
}
catch (IndexOutOfBoundsException e)
{
throw new IndexOutOfBoundsException("FullData ID Map out of sync for pos: "+this.pos+". ID: ["+id+"] greater than the number of known ID's: ["+this.entryList.size()+"].");
}
return entry;
}
finally
{
this.readWriteLock.readLock().unlock();
}
}
/** @see FullDataPointIdMap#getEntry(int) */
public IBiomeWrapper getBiomeWrapper(int id) throws IndexOutOfBoundsException { return this.getEntry(id).biome; }
/** @see FullDataPointIdMap#getEntry(int) */
public IBlockStateWrapper getBlockStateWrapper(int id) throws IndexOutOfBoundsException { return this.getEntry(id).blockState; }
/** @throws IndexOutOfBoundsException if the given ID isn't in the {@link FullDataPointIdMap#entryList} */
private Entry getEntry(int id) throws IndexOutOfBoundsException
{
Entry entry;
try
{
entry = this.entryList.get(id);
}
catch (IndexOutOfBoundsException e)
{
throw new IndexOutOfBoundsException("FullData ID Map out of sync for pos: "+this.pos+". ID: ["+id+"] greater than the number of known ID's: ["+this.entryList.size()+"].");
}
return entry;
}
/** @return -1 if the list is empty */
@@ -137,74 +124,37 @@ public class FullDataPointIdMap
* If an entry with the given values already exists nothing will
* be added but the existing item's ID will still be returned.
*/
public int addIfNotPresentAndGetId(IBiomeWrapper biome, IBlockStateWrapper blockState) { return this.addIfNotPresentAndGetId(Entry.getEntry(biome, blockState), true); }
/** @param useWriteLocks should only be false if this method is already in a write lock to prevent unlocking at the wrong time */
private int addIfNotPresentAndGetId(Entry biomeBlockStateEntry, boolean useWriteLocks)
public int addIfNotPresentAndGetId(IBiomeWrapper biome, IBlockStateWrapper blockState) { return this.addIfNotPresentAndGetId(Entry.getEntry(biome, blockState)); }
private int addIfNotPresentAndGetId(Entry biomeBlockStateEntry)
{
try
// try getting the existing ID
Integer nullableId = this.idMap.get(biomeBlockStateEntry);
if (nullableId != null)
{
if (useWriteLocks)
{
this.readWriteLock.writeLock().lock();
}
int id;
if (this.idMap.containsKey(biomeBlockStateEntry))
{
// use the existing ID
id = this.idMap.get(biomeBlockStateEntry);
}
else
{
// Add the new ID
id = this.entryList.size();
this.entryList.add(biomeBlockStateEntry);
this.idMap.put(biomeBlockStateEntry, id);
// invalidate the cached hash code
this.cachedHashCode = 0;
}
return id;
return nullableId;
}
finally
// create the new ID
return this.idMap.compute(biomeBlockStateEntry, (Entry newBiomeBlockStateEntry, Integer currentId) ->
{
if (useWriteLocks)
if (currentId != null)
{
this.readWriteLock.writeLock().unlock();
}
}
}
/** allows for adding duplicate {@link Entry} */
private void add(Entry biomeBlockStateEntry, boolean useWriteLocks)
{
try
{
if (useWriteLocks)
{
this.readWriteLock.writeLock().lock();
return currentId;
}
int id = this.entryList.size();
// Add the new ID
currentId = this.entryList.size();
this.entryList.add(biomeBlockStateEntry);
this.idMap.put(biomeBlockStateEntry, id);
// invalidate the cached hash code
this.cachedHashCode = 0;
}
finally
{
if (useWriteLocks)
{
this.readWriteLock.writeLock().unlock();
}
}
return currentId;
});
}
/**
* Adds every {@link Entry} from inputMap into this map. <br>
* Allows duplicate entries. <br><br>
@@ -216,28 +166,23 @@ public class FullDataPointIdMap
*/
public void addAll(FullDataPointIdMap inputMap)
{
try
ArrayList<Entry> entriesToMerge = inputMap.entryList;
for (int i = 0; i < entriesToMerge.size(); i++)
{
//LOGGER.trace("adding {" + this.pos + ", " + this.entryList.size() + "} and {" + inputMap.pos + ", " + inputMap.entryList.size() + "}");
inputMap.readWriteLock.readLock().lock();
this.readWriteLock.writeLock().lock();
ArrayList<Entry> entriesToMerge = inputMap.entryList;
for (int i = 0; i < entriesToMerge.size(); i++)
{
Entry entity = entriesToMerge.get(i);
this.add(entity, false);
}
}
finally
{
this.readWriteLock.writeLock().unlock();
inputMap.readWriteLock.readLock().unlock();
//LOGGER.trace("finished merging {" + this.pos + ", " + this.entryList.size() + "} and {" + inputMap.pos + ", " + inputMap.entryList.size() + "}");
Entry entity = entriesToMerge.get(i);
this.add(entity);
}
}
/** allows for adding duplicate {@link Entry} */
private void add(Entry biomeBlockStateEntry)
{
int id = this.entryList.size();
this.entryList.add(biomeBlockStateEntry);
this.idMap.put(biomeBlockStateEntry, id);
// invalidate the cached hash code
this.cachedHashCode = 0;
}
/**
* Adds each entry from the given map to this map. <br><br>
@@ -250,31 +195,16 @@ public class FullDataPointIdMap
*/
public int[] mergeAndReturnRemappedEntityIds(FullDataPointIdMap inputMap)
{
try
ArrayList<Entry> entriesToMerge = inputMap.entryList;
int[] remappedEntryIds = new int[entriesToMerge.size()];
for (int i = 0; i < entriesToMerge.size(); i++)
{
//LOGGER.trace("merging {" + this.pos + ", " + this.entryList.size() + "} and {" + inputMap.pos + ", " + inputMap.entryList.size() + "}");
inputMap.readWriteLock.readLock().lock();
this.readWriteLock.writeLock().lock();
ArrayList<Entry> entriesToMerge = inputMap.entryList;
int[] remappedEntryIds = new int[entriesToMerge.size()];
for (int i = 0; i < entriesToMerge.size(); i++)
{
Entry entity = entriesToMerge.get(i);
int id = this.addIfNotPresentAndGetId(entity, false);
remappedEntryIds[i] = id;
}
return remappedEntryIds;
}
finally
{
this.readWriteLock.writeLock().unlock();
inputMap.readWriteLock.readLock().unlock();
//LOGGER.trace("finished merging {" + this.pos + ", " + this.entryList.size() + "} and {" + inputMap.pos + ", " + inputMap.entryList.size() + "}");
Entry entity = entriesToMerge.get(i);
int id = this.addIfNotPresentAndGetId(entity);
remappedEntryIds[i] = id;
}
return remappedEntryIds;
}
/** Should only be used if this map is going to be reused, otherwise bad things will happen. */
@@ -295,38 +225,29 @@ public class FullDataPointIdMap
/** Serializes all contained entries into the given stream, formatted in UTF */
public void serialize(DhDataOutputStream outputStream) throws IOException
{
try
outputStream.writeInt(this.entryList.size());
// only used when debugging
HashMap<String, FullDataPointIdMap.Entry> dataPointEntryBySerialization = new HashMap<>();
for (Entry entry : this.entryList)
{
this.readWriteLock.readLock().lock();
outputStream.writeInt(this.entryList.size());
String entryString = entry.serialize();
outputStream.writeUTF(entryString);
// only used when debugging
HashMap<String, FullDataPointIdMap.Entry> dataPointEntryBySerialization = new HashMap<>();
for (Entry entry : this.entryList)
if (RUN_SERIALIZATION_DUPLICATE_VALIDATION)
{
String entryString = entry.serialize();
outputStream.writeUTF(entryString);
if (RUN_SERIALIZATION_DUPLICATE_VALIDATION)
if (dataPointEntryBySerialization.containsKey(entryString))
{
if (dataPointEntryBySerialization.containsKey(entryString))
{
LOGGER.error("Duplicate serialized entry found with serial: " + entryString);
}
if (dataPointEntryBySerialization.containsValue(entry))
{
LOGGER.error("Duplicate serialized entry found with value: " + entry.serialize());
}
dataPointEntryBySerialization.put(entryString, entry);
LOGGER.error("Duplicate serialized entry found with serial: " + entryString);
}
if (dataPointEntryBySerialization.containsValue(entry))
{
LOGGER.error("Duplicate serialized entry found with value: " + entry.serialize());
}
dataPointEntryBySerialization.put(entryString, entry);
}
}
finally
{
this.readWriteLock.readLock().unlock();
//LOGGER.trace("serialize " + this.pos + " " + this.entryList.size());
}
}
/** Creates a new IdBiomeBlockStateMap from the given UTF formatted stream */
@@ -429,14 +350,15 @@ public class FullDataPointIdMap
{
private static final IWrapperFactory WRAPPER_FACTORY = SingletonInjector.INSTANCE.get(IWrapperFactory.class);
private static final Int2ReferenceOpenHashMap<ArrayList<Entry>> ENTRY_POOL = new Int2ReferenceOpenHashMap<>();
/** lock is necessary since {@link Int2ReferenceOpenHashMap} isn't concurrent and concurrent threads can cause infinite loops */
private static final ReentrantReadWriteLock ENTRY_POOL_LOCK = new ReentrantReadWriteLock();
/** two levels are present so we don't need to use a key object */
private static final ConcurrentHashMap<IBiomeWrapper, ConcurrentHashMap<IBlockStateWrapper, Entry>> ENTRY_BY_BLOCKSTATE_BY_BIOMEWRAPPER = new ConcurrentHashMap<>();
public final IBiomeWrapper biome;
public final IBlockStateWrapper blockState;
private Integer hashCode = null;
private int hashCode = 0;
private boolean hashGenerated = false;
private String serialString = null;
@@ -446,62 +368,21 @@ public class FullDataPointIdMap
public static Entry getEntry(IBiomeWrapper biome, IBlockStateWrapper blockState)
{
int entryHash = getHashCode(biome, blockState);
// try getting the existing entry
try
// check for existing entry
ConcurrentHashMap<IBlockStateWrapper, Entry> entryByBlockState = ENTRY_BY_BLOCKSTATE_BY_BIOMEWRAPPER.get(biome);
if (entryByBlockState != null)
{
ENTRY_POOL_LOCK.readLock().lock();
// check if an entry already exists
ArrayList<Entry> entryList = ENTRY_POOL.get(entryHash);
if (entryList != null)
Entry entry = entryByBlockState.get(blockState);
if (entry != null)
{
// at least one entry exists with this hash code
for (int i = 0; i < entryList.size(); i++)
{
Entry entry = entryList.get(i);
if (entry.biome.equals(biome) && entry.blockState.equals(blockState))
{
return entry;
}
}
// if we got here, then there was a hash collision and this entry wasn't present in the array
return entry;
}
}
finally
{
ENTRY_POOL_LOCK.readLock().unlock();
}
// no entry exists,
// create a new one
try
{
ENTRY_POOL_LOCK.writeLock().lock();
ArrayList<Entry> entryList = ENTRY_POOL.get(entryHash);
if (entryList == null)
{
// no entries exist for this hash code
// we assume that hash collisions should basically never happen,
// so the array starts with an initial capacity of 1.
// However, since collisions will eventually happen, using an arrayList prevents unexpected bugs caused by collisions.
entryList = new ArrayList<>(1);
ENTRY_POOL.put(entryHash, entryList);
}
Entry newEntry = new Entry(biome, blockState);
entryList.add(newEntry);
return newEntry;
}
finally
{
ENTRY_POOL_LOCK.writeLock().unlock();
}
// Lazily create the inner map and new Entry
return ENTRY_BY_BLOCKSTATE_BY_BIOMEWRAPPER
.computeIfAbsent(biome, newBiome -> new ConcurrentHashMap<>())
.computeIfAbsent(blockState, newBlockState -> new Entry(biome, blockState));
}
private Entry(IBiomeWrapper biome, IBlockStateWrapper blockState)
{
@@ -515,8 +396,24 @@ public class FullDataPointIdMap
// overrides //
//===========//
public static int getHashCode(Entry entry) { return getHashCode(entry.biome, entry.blockState); }
public static int getHashCode(IBiomeWrapper biome, IBlockStateWrapper blockState)
/**
* Reminder: this hash code won't always be unique, collisions can occur;
* because of that this hash shouldn't be the only unique identifier for this object.
*/
@Override
public int hashCode()
{
// cache the hash code to improve speed
if (!this.hashGenerated)
{
this.hashCode = generateHashCode(this);
this.hashGenerated = true;
}
return this.hashCode;
}
private static int generateHashCode(Entry entry) { return generateHashCode(entry.biome, entry.blockState); }
private static int generateHashCode(IBiomeWrapper biome, IBlockStateWrapper blockState)
{
final int prime = 31;
@@ -527,26 +424,19 @@ public class FullDataPointIdMap
result = prime * result + (blockState == null ? 0 : blockState.hashCode());
return result;
}
@Override
public int hashCode()
{
// cache the hash code to improve speed
if (this.hashCode == null)
{
this.hashCode = getHashCode(this);
}
return this.hashCode;
}
@Override
public boolean equals(Object otherObj)
{
if (otherObj == this)
{
return true;
}
if (!(otherObj instanceof Entry))
{
return false;
}
Entry other = (Entry) otherObj;
return other.biome.getSerialString().equals(this.biome.getSerialString())
@@ -562,7 +452,15 @@ public class FullDataPointIdMap
// (de)serializing //
//=================//
public String serialize() { return this.biome.getSerialString() + BLOCK_STATE_SEPARATOR_STRING + this.blockState.getSerialString(); }
public String serialize()
{
if (this.serialString == null)
{
this.serialString = this.biome.getSerialString() + BLOCK_STATE_SEPARATOR_STRING + this.blockState.getSerialString();
}
return this.serialString;
}
public static Entry deserialize(String str, ILevelWrapper levelWrapper) throws DataCorruptedException
{
@@ -73,7 +73,7 @@ public class FullDataSourceV2
public static final byte DATA_FORMAT_VERSION = 1;
public static final PhantomArrayListPool ARRAY_LIST_POOL = new PhantomArrayListPool("FullDataV2", false);
public static final PhantomArrayListPool ARRAY_LIST_POOL = new PhantomArrayListPool("FullDataV2");
@@ -116,7 +116,12 @@ public class FullDataSourceV2
public final LongArrayList[] dataPoints;
public boolean isEmpty;
public boolean applyToParent = false;
/** Will be null if we don't want to update this value in the DB */
@Nullable
public Boolean applyToParent = null;
/** Will be null if we don't want to update this value in the DB */
@Nullable
public Boolean applyToChildren = null;
/** should only be used by methods exposed via the DH API */
private boolean runApiChunkValidation = false;
@@ -269,11 +274,6 @@ public class FullDataSourceV2
{
ListUtil.clearAndSetSize(this.columnWorldCompressionMode, WIDTH * WIDTH);
}
// the pooled arrays have all been set,
// the checkout object is no longer needed
this.pooledArraysCheckout = null;
}
@@ -306,10 +306,47 @@ public class FullDataSourceV2
if (inputDetailLevel == thisDetailLevel)
{
dataChanged = this.updateFromSameDetailLevel(inputDataSource, remappedIds);
// same detail level, propagate parent/children update flags from input
if (this.applyToParent != null || inputDataSource.applyToParent != null)
{
this.applyToParent =
// copy over application flag if either are set to continue propagating
(BoolUtil.falseIfNull(this.applyToParent) || BoolUtil.falseIfNull(inputDataSource.applyToParent))
// don't propagate past the top of the tree
&& (DhSectionPos.getDetailLevel(this.pos) < AbstractDataSourceHandler.TOP_SECTION_DETAIL_LEVEL);
}
// null check to prevent setting a flag we don't want to save in the DB
if (this.applyToChildren != null || inputDataSource.applyToChildren != null)
{
this.applyToChildren =
(BoolUtil.falseIfNull(this.applyToChildren) || BoolUtil.falseIfNull(inputDataSource.applyToChildren))
// don't propagate past the bottom of the tree
&& (DhSectionPos.getDetailLevel(this.pos) > AbstractDataSourceHandler.MIN_SECTION_DETAIL_LEVEL);
}
}
else if (inputDetailLevel + 1 == thisDetailLevel)
{
dataChanged = this.updateFromOneBelowDetailLevel(inputDataSource, remappedIds);
// propagating up, parent will need changes
this.applyToParent =
dataChanged
&& (BoolUtil.falseIfNull(this.applyToParent) || BoolUtil.falseIfNull(inputDataSource.applyToParent))
&& (DhSectionPos.getDetailLevel(this.pos) < AbstractDataSourceHandler.TOP_SECTION_DETAIL_LEVEL);
}
else if (inputDetailLevel - 1 == thisDetailLevel)
{
dataChanged = this.downsampleFromOneAboveDetailLevel(inputDataSource, remappedIds);
// propagating down, children will need changes
this.applyToChildren =
dataChanged
&& (BoolUtil.falseIfNull(this.applyToChildren) || BoolUtil.falseIfNull(inputDataSource.applyToChildren))
&& (DhSectionPos.getDetailLevel(this.pos) > AbstractDataSourceHandler.MIN_SECTION_DETAIL_LEVEL);
}
else
{
@@ -317,12 +354,9 @@ public class FullDataSourceV2
// and would lead to edge cases that don't necessarily need to be supported
// (IE what do you do when the input is smaller than a single datapoint in the receiving data source?)
// instead it's better to just percolate the updates up
throw new UnsupportedOperationException("Unsupported data source update. Expected input detail level of ["+thisDetailLevel+"] or ["+(thisDetailLevel+1)+"], received detail level ["+inputDetailLevel+"].");
throw new UnsupportedOperationException("Unsupported data source update. Expected input detail level of ["+(thisDetailLevel-1)+"], ["+thisDetailLevel+"], or ["+(thisDetailLevel+1)+"], received detail level ["+inputDetailLevel+"].");
}
// determine if this data source should be applied to its parent
this.applyToParent = (dataChanged && DhSectionPos.getDetailLevel(this.pos) < AbstractDataSourceHandler.TOP_SECTION_DETAIL_LEVEL);
if (dataChanged)
{
// update the hash code
@@ -331,6 +365,7 @@ public class FullDataSourceV2
return dataChanged;
}
public boolean updateFromSameDetailLevel(FullDataSourceV2 inputDataSource, int[] remappedIds)
{
// both data sources should have the same detail level
@@ -353,9 +388,31 @@ public class FullDataSourceV2
{
byte thisGenState = this.columnGenerationSteps.getByte(index);
byte inputGenState = inputDataSource.columnGenerationSteps.getByte(index);
if (inputGenState != EDhApiWorldGenerationStep.EMPTY.value
&& thisGenState <= inputGenState)
// determine if this column should be updated
boolean genStateAllowsUpdating = false;
// if the input is downsampled, we only want to replace empty or downsampled values
if (inputGenState == EDhApiWorldGenerationStep.DOWN_SAMPLED.value
&&
(
thisGenState == EDhApiWorldGenerationStep.EMPTY.value
|| thisGenState == EDhApiWorldGenerationStep.DOWN_SAMPLED.value
))
{
genStateAllowsUpdating = true;
}
// if the input is any other non-empty value,
// replace anything that is less-complete
else if (inputGenState != EDhApiWorldGenerationStep.EMPTY.value
&& thisGenState <= inputGenState)
{
// don't apply less-complete generation data
genStateAllowsUpdating = true;
}
if (genStateAllowsUpdating)
{
// check if the data changed
if (this.dataPoints[index] == null)
@@ -835,6 +892,101 @@ public class FullDataSourceV2
return value;
}
/**
* Only downsamples into a given column if this data source doesn't
* already contain data in that column.
* This is done to prevent accidentally downsampling onto already present higher-detail data.
*/
public boolean downsampleFromOneAboveDetailLevel(FullDataSourceV2 inputDataSource, int[] remappedIds)
{
if (DhSectionPos.getDetailLevel(inputDataSource.pos) - 1 != DhSectionPos.getDetailLevel(this.pos))
{
throw new IllegalArgumentException("Input data source must be exactly 1 detail level above this data source. Expected [" + (DhSectionPos.getDetailLevel(this.pos) - 1) + "], received [" + DhSectionPos.getDetailLevel(inputDataSource.pos) + "].");
}
// input is one detail level higher (lower detail)
// so 1x1 input data points will be converted into 2x2 recipient data point
// determine where in this data source should be read from
// since the input is one detail level above this will be one of input position's 4 children
int minParentXPos = DhSectionPos.getX(DhSectionPos.getChildByIndex(inputDataSource.pos, 0));
int inputOffsetX = (DhSectionPos.getX(this.pos) == minParentXPos) ? 0 : (WIDTH / 2);
int minParentZPos = DhSectionPos.getZ(DhSectionPos.getChildByIndex(inputDataSource.pos, 0));
int inputOffsetZ = (DhSectionPos.getZ(this.pos) == minParentZPos) ? 0 : (WIDTH / 2);
// merge the input's data points
// into this data source's
boolean dataChanged = false;
for (int x = 0; x < WIDTH; x++)
{
for (int z = 0; z < WIDTH; z++)
{
// recipient index is 1-to-1
int recipientIndex = relativePosToIndex(x, z);
int inputX = (x / 2) + inputOffsetX;
int inputZ = (z / 2) + inputOffsetZ;
int inputIndex = relativePosToIndex(inputX, inputZ);
// world gen //
// a separate generation step needs to be used so can replace
// this data with higher-quality data when it is available
byte inputGenStep = EDhApiWorldGenerationStep.DOWN_SAMPLED.value;
this.columnGenerationSteps.set(recipientIndex, inputGenStep);
// world compression //
byte worldCompressionMode = inputDataSource.columnWorldCompressionMode.getByte(recipientIndex);
this.columnWorldCompressionMode.set(recipientIndex, worldCompressionMode);
// data points //
// check if this column should be downsampled
boolean downSampleColumn;
if (this.dataPoints[recipientIndex] == null)
{
downSampleColumn = true;
}
else
{
downSampleColumn = true; // assume empty until we find non-empty data
for (long dataPoint : this.dataPoints[recipientIndex])
{
if (dataPoint != FullDataPointUtil.EMPTY_DATA_POINT)
{
downSampleColumn = false;
break;
}
}
}
if (downSampleColumn)
{
LongArrayList inputDataArray = inputDataSource.dataPoints[inputIndex];
this.dataPoints[recipientIndex] = inputDataArray;
this.remapDataColumn(recipientIndex, remappedIds);
if (RUN_DATA_ORDER_VALIDATION)
{
throwIfDataColumnInWrongOrder(inputDataSource.pos, this.dataPoints[recipientIndex]);
}
dataChanged = true;
}
this.isEmpty = false;
}
}
return dataChanged;
}
//================//
@@ -866,6 +1018,11 @@ public class FullDataSourceV2
*/
public static void throwIfDataColumnInWrongOrder(long pos, LongArrayList dataArray) throws IllegalStateException
{
if (dataArray.size() < 2)
{
return;
}
long firstDataPoint = dataArray.getLong(0);
int firstBottomY = FullDataPointUtil.getBottomY(firstDataPoint);
@@ -884,6 +1041,11 @@ public class FullDataSourceV2
*/
private static void ensureDataColumnOrder(LongArrayList dataColumn)
{
if (dataColumn.size() < 2)
{
return;
}
long firstDataPoint = dataColumn.getLong(0);
int firstBottomY = FullDataPointUtil.getBottomY(firstDataPoint);
@@ -972,7 +1134,7 @@ public class FullDataSourceV2
LongArrayList packedDataPoints = LodDataBuilder.convertApiDataPointListToPackedLongArray(columnDataPoints, this, 0);
// TODO there should be an "unknown" compression and generation step, or be defined via the datapoints
this.setSingleColumn(packedDataPoints, relX, relZ, EDhApiWorldGenerationStep.LIGHT, EDhApiWorldCompressionMode.MERGE_SAME_BLOCKS);
this.setSingleColumn(packedDataPoints, relX, relZ, EDhApiWorldGenerationStep.SURFACE, EDhApiWorldCompressionMode.MERGE_SAME_BLOCKS);
return columnDataPoints;
}
@@ -0,0 +1,101 @@
package com.seibel.distanthorizons.core.dataObjects.render;
import com.google.common.cache.Cache;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.dataObjects.transformers.FullDataToRenderDataTransformer;
import com.seibel.distanthorizons.core.file.fullDatafile.FullDataSourceProviderV2;
import com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker;
import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantLock;
/**
* Wrapper for {@link ColumnRenderSource} that handles reference counting
* and cache tracking.
*/
public class CachedColumnRenderSource implements AutoCloseable
{
/** an externally handled future that will complete once the {@link CachedColumnRenderSource#columnRenderSource} has finished loading */
public final CompletableFuture<CachedColumnRenderSource> loadFuture;
/** will be null initially, should be non-null once the corresponding load future is done */
@Nullable
public ColumnRenderSource columnRenderSource = null;
private final AtomicInteger referenceCount;
private final Cache<Long, CachedColumnRenderSource> cachedRenderSourceByPos;
private final ReentrantLock getterLock;
//=============//
// constructor //
//=============//
public CachedColumnRenderSource(
@NotNull CompletableFuture<CachedColumnRenderSource> loadFuture,
@NotNull ReentrantLock getterLock,
@NotNull Cache<Long, CachedColumnRenderSource> cachedRenderSourceByPos)
{
this.loadFuture = loadFuture;
this.getterLock = getterLock;
this.referenceCount = new AtomicInteger(1);
this.cachedRenderSourceByPos = cachedRenderSourceByPos;
}
//====================//
// reference counting //
//====================//
public void markInUse() { this.referenceCount.getAndIncrement(); }
//================//
// base overrides //
//================//
/**
* Will be called multiple times,
* however it will only close the underlying data once
* all references have closed.
*/
@Override
public void close() throws IllegalStateException
{
try
{
// lock to prevent other threads for accessing the cache if we invalidate it
this.getterLock.lock();
// should only happen if something goes wrong up-stream
if (this.columnRenderSource == null)
{
return;
}
// only close once everyone is done with this datasource
int refCount = this.referenceCount.decrementAndGet();
if (refCount == 0)
{
this.cachedRenderSourceByPos.invalidate(this.columnRenderSource.pos);
this.columnRenderSource.close();
}
else if (refCount < 0)
{
throw new IllegalStateException("Render source ["+this.columnRenderSource.pos+"] reference count incorrect. Object already closed.");
}
}
finally
{
this.getterLock.unlock();
}
}
}
@@ -41,8 +41,6 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.ThreadPoolExecutor;
/**
* Used to populate the buffers in a {@link ColumnRenderSource} object.
@@ -139,7 +139,7 @@ public class FullDataToRenderDataTransformer
}
columnSource.fillDebugFlag(0, 0, ColumnRenderSource.SECTION_SIZE, ColumnRenderSource.SECTION_SIZE, ColumnRenderSource.DebugSourceFlag.FULL);
return columnSource;
}
@@ -270,7 +270,7 @@ public class FullDataToRenderDataTransformer
//====================//
boolean ignoreBlock = blockStatesToIgnore.contains(block);
boolean caveBlock = caveBlockStatesToIgnore.contains(block);
boolean caveBlock = caveBlockStatesToIgnore.contains(block); // TODO caves should also ignore transparent/non-solid blocks (IE grass and plants) wthout each being defined
if (caveBlock)
{
if (caveCullingEnabled
@@ -65,60 +65,44 @@ public class LodDataBuilder
// only block lighting is needed here, sky lighting is populated at the data source stage
LodUtil.assertTrue(chunkWrapper.isDhBlockLightingCorrect());
int sectionPosX = getXOrZSectionPosFromChunkPos(chunkWrapper.getChunkPos().getX());
int sectionPosZ = getXOrZSectionPosFromChunkPos(chunkWrapper.getChunkPos().getZ());
long pos = DhSectionPos.encode(DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL, sectionPosX, sectionPosZ);
FullDataSourceV2 dataSource = FullDataSourceV2.createEmpty(pos);
dataSource.isEmpty = false;
// chunk updates always propagate up
dataSource.applyToParent = true;
// compute the chunk dataSource offset
// this offset is used to determine where in the dataSource this chunk's data should go
int chunkOffsetX = chunkWrapper.getChunkPos().getX();
if (chunkWrapper.getChunkPos().getX() < 0)
{
// expected offset positions:
// chunkPos -> offset
// 5 -> 1
// 4 -> 0 ---
// 3 -> 3
// 2 -> 2
// 1 -> 1
// 0 -> 0 ===
// -1 -> 3
// -2 -> 2
// -3 -> 1
// -4 -> 0 ---
// -5 -> 3
chunkOffsetX = ((chunkOffsetX) % FullDataSourceV2.NUMB_OF_CHUNKS_WIDE);
if (chunkOffsetX != 0)
{
chunkOffsetX += FullDataSourceV2.NUMB_OF_CHUNKS_WIDE;
}
}
else
{
chunkOffsetX %= FullDataSourceV2.NUMB_OF_CHUNKS_WIDE;
}
chunkOffsetX *= LodUtil.CHUNK_WIDTH;
int chunkOffsetZ = chunkWrapper.getChunkPos().getZ();
if (chunkWrapper.getChunkPos().getZ() < 0)
{
chunkOffsetZ = ((chunkOffsetZ) % FullDataSourceV2.NUMB_OF_CHUNKS_WIDE);
if (chunkOffsetZ != 0)
{
chunkOffsetZ += FullDataSourceV2.NUMB_OF_CHUNKS_WIDE;
}
}
else
{
chunkOffsetZ %= FullDataSourceV2.NUMB_OF_CHUNKS_WIDE;
}
// expected offset positions:
// chunkPos -> offset
// 5 -> 1
// 4 -> 0 ---
// 3 -> 3
// 2 -> 2
// 1 -> 1
// 0 -> 0 ===
// -1 -> 3
// -2 -> 2
// -3 -> 1
// -4 -> 0 ---
// -5 -> 3
// Fast modulo calculation using bitwise AND since NUMB_OF_CHUNKS_WIDE is a power of 2 (4)
// For any number n: n & (2^k - 1) is equivalent to Math.floorMod(n, 2^k)
// Original: Math.floorMod(x, 4) - Handles negative numbers, gives non-negative result in range [0,3]
// Bitwise: x & (4-1) - Also gives non-negative result in range [0,3]
// Example: -5 & 3 = 3, which equals Math.floorMod(-5, 4) = 3
int chunkOffsetX = chunkWrapper.getChunkPos().getX() & (FullDataSourceV2.NUMB_OF_CHUNKS_WIDE - 1);
int chunkOffsetZ = chunkWrapper.getChunkPos().getZ() & (FullDataSourceV2.NUMB_OF_CHUNKS_WIDE - 1);
// Convert from chunk coordinates to block coordinates
chunkOffsetX *= LodUtil.CHUNK_WIDTH;
chunkOffsetZ *= LodUtil.CHUNK_WIDTH;
@@ -136,49 +120,49 @@ public class LodDataBuilder
IBlockStateWrapper previousBlockState = null;
int minBuildHeight = chunkWrapper.getMinNonEmptyHeight();
int exclusiveMaxBuildHeight = chunkWrapper.getExclusiveMaxBuildHeight();
int inclusiveMinBuildHeight = chunkWrapper.getInclusiveMinBuildHeight();
int dataCapacity = chunkWrapper.getHeight() / 4;
for (int relBlockX = 0; relBlockX < LodUtil.CHUNK_WIDTH; relBlockX++)
{
for (int relBlockZ = 0; relBlockZ < LodUtil.CHUNK_WIDTH; relBlockZ++)
{
LongArrayList longs = dataSource.get(
relBlockX + chunkOffsetX,
relBlockZ + chunkOffsetZ);
// Calculate column position
int columnX = relBlockX + chunkOffsetX;
int columnZ = relBlockZ + chunkOffsetZ;
// Get column data
LongArrayList longs = dataSource.get(columnX, columnZ);
if (longs == null)
{
longs = new LongArrayList(chunkWrapper.getHeight() / 4);
longs = new LongArrayList(dataCapacity);
}
else
{
longs.clear();
}
int lastY = chunkWrapper.getExclusiveMaxBuildHeight();
int lastY = exclusiveMaxBuildHeight;
IBiomeWrapper biome = chunkWrapper.getBiome(relBlockX, lastY, relBlockZ);
IBlockStateWrapper blockState = AIR;
int mappedId = dataSource.mapping.addIfNotPresentAndGetId(biome, blockState);
// Determine lighting (we are at the height limit. There are no torches here, and sky is not obscured.) // TODO: Per face lighting someday?
byte blockLight = LodUtil.MIN_MC_LIGHT;
byte skyLight = LodUtil.MAX_MC_LIGHT;
byte blockLight;
byte skyLight;
if (lastY < chunkWrapper.getExclusiveMaxBuildHeight())
{
// FIXME: The lastY +1 offset is to reproduce the old behavior. Remove this when we get per-face lighting
blockLight = (byte) chunkWrapper.getDhBlockLight(relBlockX, lastY + 1, relBlockZ);
skyLight = (byte) chunkWrapper.getDhSkyLight(relBlockX, lastY + 1, relBlockZ);
}
else
{
//we are at the height limit. There are no torches here, and sky is not obscured.
blockLight = LodUtil.MIN_MC_LIGHT;
skyLight = LodUtil.MAX_MC_LIGHT;
}
// Get the maximum height from both heightmaps
int y = Math.max(
// max between both heightmaps to account for solid invisible blocks (glass)
// and non-solid opaque blocks (at one point this was stairs, not sure what would fit this now)
chunkWrapper.getLightBlockingHeightMapValue(relBlockX, relBlockZ),
chunkWrapper.getSolidHeightMapValue(relBlockX, relBlockZ)
);
// determine the starting Y Pos
int y = chunkWrapper.getLightBlockingHeightMapValue(relBlockX, relBlockZ);
// go up until we reach open air or the world limit
// Go up until we reach open air or the world limit
IBlockStateWrapper topBlockState = previousBlockState = chunkWrapper.getBlockState(relBlockX, y, relBlockZ, mcBlockPos, previousBlockState);
while (!topBlockState.isAir() && y < chunkWrapper.getExclusiveMaxBuildHeight())
while (!topBlockState.isAir() && y < exclusiveMaxBuildHeight)
{
try
{
@@ -191,7 +175,7 @@ public class LodDataBuilder
{
if (!getTopErrorLogged)
{
LOGGER.warn("Unexpected issue in LodDataBuilder, future errors won't be logged. Chunk [" + chunkWrapper.getChunkPos() + "] with max height: [" + chunkWrapper.getExclusiveMaxBuildHeight() + "] had issue getting block at pos [" + relBlockX + "," + y + "," + relBlockZ + "] error: " + e.getMessage(), e);
LOGGER.warn("Unexpected issue in LodDataBuilder, future errors won't be logged. Chunk [" + chunkWrapper.getChunkPos() + "] with max height: [" + exclusiveMaxBuildHeight + "] had issue getting block at pos [" + relBlockX + "," + y + "," + relBlockZ + "] error: " + e.getMessage(), e);
getTopErrorLogged = true;
}
@@ -200,7 +184,7 @@ public class LodDataBuilder
}
}
// Process blocks from top to bottom
for (; y >= minBuildHeight; y--)
{
IBiomeWrapper newBiome = chunkWrapper.getBiome(relBlockX, y, relBlockZ);
@@ -208,10 +192,10 @@ public class LodDataBuilder
byte newBlockLight = (byte) chunkWrapper.getDhBlockLight(relBlockX, y + 1, relBlockZ);
byte newSkyLight = (byte) chunkWrapper.getDhSkyLight(relBlockX, y + 1, relBlockZ);
// save the biome/block change
// Save the biome/block change if different from previous
if (!newBiome.equals(biome) || !newBlockState.equals(blockState))
{
longs.add(FullDataPointUtil.encode(mappedId, lastY - y, y + 1 - chunkWrapper.getInclusiveMinBuildHeight(), blockLight, skyLight));
longs.add(FullDataPointUtil.encode(mappedId, lastY - y, y + 1 - inclusiveMinBuildHeight, blockLight, skyLight));
biome = newBiome;
blockState = newBlockState;
@@ -221,17 +205,16 @@ public class LodDataBuilder
lastY = y;
}
}
longs.add(FullDataPointUtil.encode(mappedId, lastY - y, y + 1 - chunkWrapper.getInclusiveMinBuildHeight(), blockLight, skyLight));
dataSource.setSingleColumn(longs,
relBlockX + chunkOffsetX,
relBlockZ + chunkOffsetZ,
EDhApiWorldGenerationStep.LIGHT,
worldCompressionMode);
// Add the final data point
longs.add(FullDataPointUtil.encode(mappedId, lastY - y, y + 1 - inclusiveMinBuildHeight, blockLight, skyLight));
// Set the column in the data source
dataSource.setSingleColumn(longs, columnX, columnZ, EDhApiWorldGenerationStep.LIGHT, worldCompressionMode);
}
}
if (ignoreHiddenBlocks)
if (ignoreHiddenBlocks)
{
cullHiddenBlocks(dataSource, chunkOffsetX, chunkOffsetZ);
}
@@ -268,16 +251,16 @@ public class LodDataBuilder
{
long currentPoint = centerColumn.getLong(centerIndex);
// translucent data points are not eligible to be culled.
// Translucent data points are not eligible to be culled.
if (isTranslucent(dataSource, currentPoint))
{
continue;
}
// the top segment should never be culled.
if (centerIndex == 0
|| isTranslucent(dataSource, centerColumn.getLong(centerIndex - 1))
)
if (centerIndex == 0
|| isTranslucent(dataSource, centerColumn.getLong(centerIndex - 1))
)
{
continue;
}
@@ -285,9 +268,9 @@ public class LodDataBuilder
// the bottom segment can sometimes be culled.
// assume it will not be seen from below,
// because this would imply the player is in the void.
if (centerIndex + 1 < centerColumn.size()
&& isTranslucent(dataSource, centerColumn.getLong(centerIndex + 1))
)
if (centerIndex + 1 < centerColumn.size()
&& isTranslucent(dataSource, centerColumn.getLong(centerIndex + 1))
)
{
continue;
}
@@ -320,9 +303,11 @@ public class LodDataBuilder
continue;
}
// current point is fully surrounded. remove it.
// Current point is fully surrounded. remove it.
centerColumn.removeLong(centerIndex);
// make the above data point cover the area that the current point used to occupy.
// Make the above data point cover the area that the current point used to occupy.
// The element that was at `centerIndex - 1` is still at that position even after removal of centerIndex.
long above = centerColumn.getLong(centerIndex - 1);
above = FullDataPointUtil.setBottomY(above, FullDataPointUtil.getBottomY(currentPoint));
above = FullDataPointUtil.setHeight(above, FullDataPointUtil.getHeight(currentPoint) + FullDataPointUtil.getHeight(above));
@@ -331,31 +316,31 @@ public class LodDataBuilder
}
}
}
/**
checks if centerPoint is "covered" by opaque data points in adjacentColumn.
centerPoint counts as covered if, and only if, for all Y levels in its height range,
there exists an opaque data point in adjacentColumn which overlaps with that Y level.
@param source used to lookup blocks (and their opacities) based on their IDs.
@param centerPoint the point being checked to see if it's fully covered.
@param adjacentColumn the data points which might cover centerPoint.
@param adjacentIndex the starting index in adjacentColumn to start scanning at.
indices greater than adjacentIndex have already been checked and confirmed to
not overlap or only overlap partially with centerPoint's Y range.
@return if centerPoint is covered, returns the index of the segment which finishes covering it.
the start of the covering may be a smaller index. in this case, the returned index may be used
as the adjacentIndex provided to this method on the next iteration which yields a new centerPoint.
if centerPoint is NOT covered, returns the bitwise negation of the index of the
segment which did not cover it. this guarantees that the returned value is negative.
the caller should check for negative return values and manually un-negate them to proceed with the loop.
in other words, this function returns the index of the next adjacent data
point to use in the loop, AND a boolean indicating whether or not the
centerPoint is covered; both are packed into the same int, and returned.
*/
checks if centerPoint is "covered" by opaque data points in adjacentColumn.
centerPoint counts as covered if, and only if, for all Y levels in its height range,
there exists an opaque data point in adjacentColumn which overlaps with that Y level.
@param source used to lookup blocks (and their opacities) based on their IDs.
@param centerPoint the point being checked to see if it's fully covered.
@param adjacentColumn the data points which might cover centerPoint.
@param adjacentIndex the starting index in adjacentColumn to start scanning at.
indices greater than adjacentIndex have already been checked and confirmed to
not overlap or only overlap partially with centerPoint's Y range.
@return if centerPoint is covered, returns the index of the segment which finishes covering it.
the start of the covering may be a smaller index. in this case, the returned index may be used
as the adjacentIndex provided to this method on the next iteration which yields a new centerPoint.
if centerPoint is NOT covered, returns the bitwise negation of the index of the
segment which did not cover it. this guarantees that the returned value is negative.
the caller should check for negative return values and manually un-negate them to proceed with the loop.
in other words, this function returns the index of the next adjacent data
point to use in the loop, AND a boolean indicating whether or not the
centerPoint is covered; both are packed into the same int, and returned.
*/
private static int checkOcclusion(FullDataSourceV2 source, long centerPoint, LongArrayList adjacentColumn, int adjacentIndex)
{
int bottomOfCenter = FullDataPointUtil.getBottomY(centerPoint);
@@ -380,12 +365,12 @@ public class LodDataBuilder
throw new LodUtil.AssertFailureException("Adjacent column ends before center column does.");
}
private static boolean isTranslucent(FullDataSourceV2 source, long point) {
return source.mapping.getBlockStateWrapper(FullDataPointUtil.getId(point)).getOpacity() < LodUtil.BLOCK_FULLY_OPAQUE;
}
/** @throws ClassCastException if an API user returns the wrong object type(s) */
public static FullDataSourceV2 createFromApiChunkData(DhApiChunk apiChunk, boolean runAdditionalValidation) throws ClassCastException, DataCorruptedException, IllegalArgumentException
@@ -395,9 +380,13 @@ public class LodDataBuilder
int sectionPosZ = getXOrZSectionPosFromChunkPos(apiChunk.chunkPosZ);
long pos = DhSectionPos.encode(DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL, sectionPosX, sectionPosZ);
// chunk relative block position in the data source
int relSourceBlockX = Math.floorMod(apiChunk.chunkPosX, 4) * LodUtil.CHUNK_WIDTH;
int relSourceBlockZ = Math.floorMod(apiChunk.chunkPosZ, 4) * LodUtil.CHUNK_WIDTH;
// Fast modulo calculation using bitwise AND since NUMB_OF_CHUNKS_WIDE is a power of 2 (4)
// For any number n: n & (2^k - 1) is equivalent to Math.floorMod(n, 2^k)
// Original: Math.floorMod(x, 4) - Handles negative numbers, gives non-negative result in range [0,3]
// Bitwise: x & (4-1) - Also gives non-negative result in range [0,3]
// Example: -5 & 3 = 3, which equals Math.floorMod(-5, 4) = 3
int relSourceBlockX = (apiChunk.chunkPosX & (FullDataSourceV2.NUMB_OF_CHUNKS_WIDE - 1)) * LodUtil.CHUNK_WIDTH;
int relSourceBlockZ = (apiChunk.chunkPosZ & (FullDataSourceV2.NUMB_OF_CHUNKS_WIDE - 1)) * LodUtil.CHUNK_WIDTH;
FullDataSourceV2 dataSource = FullDataSourceV2.createEmpty(pos);
for (int relBlockZ = 0; relBlockZ < LodUtil.CHUNK_WIDTH; relBlockZ++)
@@ -416,8 +405,8 @@ public class LodDataBuilder
// TODO add the ability for API users to define a different compression mode
// or add a "unkown" compression mode
dataSource.setSingleColumn(
packedDataPoints,
relBlockX + relSourceBlockX, relBlockZ + relSourceBlockZ,
packedDataPoints,
relBlockX + relSourceBlockX, relBlockZ + relSourceBlockZ,
EDhApiWorldGenerationStep.LIGHT, EDhApiWorldCompressionMode.MERGE_SAME_BLOCKS);
dataSource.isEmpty = false;
}
@@ -433,7 +422,7 @@ public class LodDataBuilder
/** @see FullDataPointUtil */
public static LongArrayList convertApiDataPointListToPackedLongArray(
@Nullable List<DhApiTerrainDataPoint> columnDataPoints, FullDataSourceV2 dataSource,
@Nullable List<DhApiTerrainDataPoint> columnDataPoints, FullDataSourceV2 dataSource,
int bottomYBlockPos) throws DataCorruptedException
{
// this null check does 2 nice things at the same time:
@@ -526,13 +515,13 @@ public class LodDataBuilder
}
// is there a gap between the last datapoint?
if (topYPos != lastBottomYPos
&& lastBottomYPos != Integer.MIN_VALUE)
&& lastBottomYPos != Integer.MIN_VALUE)
{
throw new IllegalArgumentException("DhApiTerrainDataPoint ["+i+"] has a gap between it and index ["+(i-1)+"]. Empty spaces should be filled by air, otherwise DH's downsampling won't calculate lighting correctly.");
}
lastBottomYPos = bottomYPos;
lastBottomYPos = bottomYPos;
}
}
@@ -1,177 +0,0 @@
package com.seibel.distanthorizons.core.file.beacon;
import com.seibel.distanthorizons.core.pos.DhChunkPos;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos;
import com.seibel.distanthorizons.core.render.renderer.generic.BeaconRenderHandler;
import com.seibel.distanthorizons.core.render.renderer.generic.GenericObjectRenderer;
import com.seibel.distanthorizons.core.sql.dto.BeaconBeamDTO;
import com.seibel.distanthorizons.core.sql.repo.BeaconBeamRepo;
import com.seibel.distanthorizons.core.util.KeyedLockContainer;
import com.seibel.distanthorizons.core.util.LodUtil;
import org.jetbrains.annotations.NotNull;
import javax.annotation.Nullable;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.concurrent.locks.ReentrantLock;
public class BeaconBeamDataHandler
{
private final BeaconBeamRepo beaconBeamRepo;
@Nullable
private BeaconRenderHandler beaconRenderHandler;
private final KeyedLockContainer<Long> updateLockContainer = new KeyedLockContainer<>();
//=============//
// constructor //
//=============//
public BeaconBeamDataHandler(@NotNull BeaconBeamRepo beaconBeamRepo, @Nullable GenericObjectRenderer renderer)
{
this.beaconBeamRepo = beaconBeamRepo;
if (renderer != null)
{
this.beaconRenderHandler = new BeaconRenderHandler(renderer);
}
}
//==========//
// updating //
//==========//
public void setBeaconBeamsForChunk(DhChunkPos chunkPos, List<BeaconBeamDTO> activeBeamList)
{
long sectionPos = DhSectionPos.encode(LodUtil.CHUNK_DETAIL_LEVEL, chunkPos.getX(), chunkPos.getZ());
this.setBeaconBeamsForPos(sectionPos, activeBeamList);
}
public void setBeaconBeamsForPos(long sectionPos, List<BeaconBeamDTO> activeBeamList)
{
// locked to prevent two threads from updating the same section at the same time
ReentrantLock lock = this.updateLockContainer.getLockForPos(sectionPos);
try
{
lock.lock();
HashSet<DhBlockPos> allPosSet = new HashSet<>();
// sort new beams
HashMap<DhBlockPos, BeaconBeamDTO> activeBeamByPos = new HashMap<>(activeBeamList.size());
for (BeaconBeamDTO beam : activeBeamList)
{
activeBeamByPos.put(beam.blockPos, beam);
allPosSet.add(beam.blockPos);
}
// get existing beams
List<BeaconBeamDTO> existingBeamList = this.beaconBeamRepo.getAllBeamsForPos(sectionPos);
HashMap<DhBlockPos, BeaconBeamDTO> existingBeamByPos = new HashMap<>(existingBeamList.size());
for (BeaconBeamDTO beam : existingBeamList)
{
existingBeamByPos.put(beam.blockPos, beam);
allPosSet.add(beam.blockPos);
}
for (DhBlockPos beaconPos : allPosSet)
{
if (!DhSectionPos.contains(sectionPos, beaconPos))
{
// don't update beacons outside the updated chunk
continue;
}
BeaconBeamDTO existingBeam = existingBeamByPos.get(beaconPos);
BeaconBeamDTO activeBeam = activeBeamByPos.get(beaconPos);
if (activeBeam != null)
{
if (existingBeam == null)
{
// new beam found, add to DB
this.beaconBeamRepo.save(activeBeam);
if (this.beaconRenderHandler != null)
{
this.beaconRenderHandler.startRenderingBeacon(activeBeam);
}
}
else
{
// beam still exists in chunk
if (!existingBeam.color.equals(activeBeam.color))
{
// beam colors were changed
this.beaconBeamRepo.save(activeBeam);
if (this.beaconRenderHandler != null)
{
this.beaconRenderHandler.updateBeaconColor(activeBeam);
}
}
}
}
else if (existingBeam != null)
{
// beam no longer exists at position, remove from DB
this.beaconBeamRepo.deleteWithKey(beaconPos);
if (this.beaconRenderHandler != null)
{
this.beaconRenderHandler.stopRenderingBeaconAtPos(beaconPos);
}
}
}
}
finally
{
lock.unlock();
}
}
//===================//
// loading/unloading //
//===================//
public void loadBeaconBeamsInPos(long pos)
{
if (this.beaconRenderHandler == null)
{
return;
}
// get all beams in pos
List<BeaconBeamDTO> existingBeamList = this.beaconBeamRepo.getAllBeamsForPos(pos);
for (BeaconBeamDTO newBeam : existingBeamList)
{
this.beaconRenderHandler.startRenderingBeacon(newBeam);
}
}
public void unloadBeaconBeamsInPos(long pos)
{
if (this.beaconRenderHandler == null)
{
return;
}
// get all beams in pos
List<BeaconBeamDTO> existingBeamList = this.beaconBeamRepo.getAllBeamsForPos(pos);
for (BeaconBeamDTO beam : existingBeamList)
{
this.beaconRenderHandler.stopRenderingBeaconAtPos(beam.blockPos);
}
}
}
@@ -106,14 +106,12 @@ public class DelayedFullDataSourceSaveCache implements AutoCloseable
}
}
public int getUnsavedCount() { return (int)this.dataSourceByPosition.size(); }
public void handleDataSourceRemoval(RemovalNotification<Long, FullDataSourceV2> removalNotification)
{
RemovalCause cause = removalNotification.getCause();
if (cause == RemovalCause.EXPIRED
|| cause == RemovalCause.COLLECTED
|| cause == RemovalCause.EXPLICIT
|| cause == RemovalCause.SIZE)
{
// close the data source after it has expired from the cache
@@ -144,6 +142,35 @@ public class DelayedFullDataSourceSaveCache implements AutoCloseable
//==============//
// List methods //
//==============//
public int getUnsavedCount() { return (int)this.dataSourceByPosition.size(); }
/** Removes everything from the memory cache and fires the {@link DelayedFullDataSourceSaveCache#onSaveTimeoutAsyncFunc} for each. */
public void flush()
{
Set<Long> keySet = this.dataSourceByPosition.asMap().keySet();
for (Long pos : keySet)
{
ReentrantLock lock = this.saveLockContainer.getLockForPos(pos);
try
{
lock.lock();
this.dataSourceByPosition.invalidate(pos);
}
finally
{
lock.unlock();
}
}
}
//================//
// static cleanup //
//================//
@@ -68,9 +68,9 @@ public class FullDataSourceProviderV2
private static final Logger LOGGER = DhLoggerBuilder.getLogger();
private static final IMinecraftClientWrapper MC_CLIENT = SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class);
protected static final int NUMBER_OF_PARENT_UPDATE_TASKS_PER_THREAD = 50;
protected static final int NUMBER_OF_PARENT_UPDATE_TASKS_PER_THREAD = 5;
/** how many parent update tasks can be in the queue at once */
protected static final int MAX_UPDATE_TASK_COUNT = NUMBER_OF_PARENT_UPDATE_TASKS_PER_THREAD * Config.Common.MultiThreading.numberOfThreads.get();
protected static int getMaxUpdateTaskCount() { return NUMBER_OF_PARENT_UPDATE_TASKS_PER_THREAD* Config.Common.MultiThreading.numberOfThreads.get(); }
/** indicates how long the update queue thread should wait between queuing ticks */
protected static final int UPDATE_QUEUE_THREAD_DELAY_IN_MS = 250;
@@ -103,7 +103,7 @@ public class FullDataSourceProviderV2
* Tracks which positions are currently being updated
* to prevent duplicate concurrent updates.
*/
public final Set<Long> parentUpdatingPosSet = ConcurrentHashMap.newKeySet();
public final Set<Long> updatingPosSet = ConcurrentHashMap.newKeySet();
// TODO only run thread if modifications happened recently
/**
@@ -225,106 +225,11 @@ public class FullDataSourceProviderV2
targetBlockPos = MC_CLIENT.getPlayerBlockPos();
}
// queue parent updates
if (executor.getQueueSize() < MAX_UPDATE_TASK_COUNT
&& this.parentUpdatingPosSet.size() < MAX_UPDATE_TASK_COUNT)
this.runParentUpdates(executor, targetBlockPos);
if (Config.Common.LodBuilding.Experimental.upsampleLowerDetailLodsToFillHoles.get())
{
// get the positions that need to be applied to their parents
LongArrayList parentUpdatePosList = this.repo.getPositionsToUpdate(targetBlockPos.getX(), targetBlockPos.getZ(), MAX_UPDATE_TASK_COUNT);
// combine updates together based on their parent
HashMap<Long, HashSet<Long>> updatePosByParentPos = new HashMap<>();
for (Long pos : parentUpdatePosList)
{
updatePosByParentPos.compute(DhSectionPos.getParentPos(pos), (parentPos, updatePosSet) ->
{
if (updatePosSet == null)
{
updatePosSet = new HashSet<>();
}
updatePosSet.add(pos);
return updatePosSet;
});
}
// queue the updates
for (Long parentUpdatePos : updatePosByParentPos.keySet())
{
// stop if there are already a bunch of updates queued
if (this.parentUpdatingPosSet.size() > MAX_UPDATE_TASK_COUNT
|| !this.parentUpdatingPosSet.add(parentUpdatePos))
{
break;
}
try
{
executor.execute(() ->
{
ReentrantLock parentWriteLock = this.updateLockProvider.getLock(parentUpdatePos);
boolean parentLocked = false;
try
{
//LOGGER.info("updating parent: "+parentUpdatePos);
// Locking the parent before the children should prevent deadlocks.
// TryLock is used instead of lock so this thread can handle a different update.
if (parentWriteLock.tryLock())
{
parentLocked = true;
this.lockedPosSet.add(parentUpdatePos);
// apply each child pos to the parent
for (Long childPos : updatePosByParentPos.get(parentUpdatePos))
{
ReentrantLock childReadLock = this.updateLockProvider.getLock(childPos);
try
{
childReadLock.lock();
this.lockedPosSet.add(childPos);
try (FullDataSourceV2 dataSource = this.get(childPos))
{
// can return null when the file handler is being shut down
if (dataSource != null)
{
this.updateDataSourceAtPos(parentUpdatePos, dataSource, false);
this.repo.setApplyToParent(childPos, false);
}
}
}
catch (Exception e)
{
LOGGER.error("Unexpected in update for parent pos: [" + DhSectionPos.toString(parentUpdatePos) + "] Error: [" + e.getMessage() + "].", e);
}
finally
{
childReadLock.unlock();
this.lockedPosSet.remove(childPos);
}
}
}
}
finally
{
if (parentLocked)
{
parentWriteLock.unlock();
this.lockedPosSet.remove(parentUpdatePos);
}
this.parentUpdatingPosSet.remove(parentUpdatePos);
}
});
}
catch (RejectedExecutionException ignore)
{ /* the executor was shut down, it should be back up shortly and able to accept new jobs */ }
catch (Exception e)
{
this.parentUpdatingPosSet.remove(parentUpdatePos);
throw e;
}
}
this.runChildUpdates(executor, targetBlockPos);
}
}
@@ -340,6 +245,248 @@ public class FullDataSourceProviderV2
LOGGER.info("Update thread ["+Thread.currentThread().getName()+"] terminated.");
}
/** will always apply updates */
private void runParentUpdates(PriorityTaskPicker.Executor executor, DhBlockPos targetBlockPos)
{
int maxUpdateTaskCount = getMaxUpdateTaskCount();
// queue parent updates
if (executor.getQueueSize() < maxUpdateTaskCount
&& this.updatingPosSet.size() < maxUpdateTaskCount)
{
// get the positions that need to be applied to their parents
LongArrayList parentUpdatePosList = this.repo.getPositionsToUpdate(targetBlockPos.getX(), targetBlockPos.getZ(), maxUpdateTaskCount);
// combine updates together based on their parent
HashMap<Long, HashSet<Long>> updatePosByParentPos = new HashMap<>();
for (Long pos : parentUpdatePosList)
{
updatePosByParentPos.compute(DhSectionPos.getParentPos(pos), (parentPos, updatePosSet) ->
{
if (updatePosSet == null)
{
updatePosSet = new HashSet<>();
}
updatePosSet.add(pos);
return updatePosSet;
});
}
// queue the updates
for (Long parentUpdatePos : updatePosByParentPos.keySet())
{
// stop if there are already a bunch of updates queued
if (this.updatingPosSet.size() > maxUpdateTaskCount
|| executor.getQueueSize() > maxUpdateTaskCount
|| !this.updatingPosSet.add(parentUpdatePos))
{
break;
}
try
{
executor.execute(() ->
{
ReentrantLock parentWriteLock = this.updateLockProvider.getLock(parentUpdatePos);
boolean parentLocked = false;
try
{
//LOGGER.info("updating parent: "+parentUpdatePos);
// Locking the parent before the children should prevent deadlocks.
// TryLock is used instead of lock so this thread can handle a different update.
if (parentWriteLock.tryLock())
{
parentLocked = true;
this.lockedPosSet.add(parentUpdatePos);
try (FullDataSourceV2 parentDataSource = this.get(parentUpdatePos))
{
// will return null if the file handler is shutting down
if (parentDataSource != null)
{
// apply each child pos to the parent
for (Long childPos : updatePosByParentPos.get(parentUpdatePos))
{
ReentrantLock childReadLock = this.updateLockProvider.getLock(childPos);
try
{
childReadLock.lock();
this.lockedPosSet.add(childPos);
try (FullDataSourceV2 childDataSource = this.get(childPos))
{
// can return null when the file handler is being shut down
if (childDataSource != null)
{
parentDataSource.update(childDataSource);
}
}
}
catch (Exception e)
{
LOGGER.error("Unexpected in parent update propagation for parent pos: ["+DhSectionPos.toString(parentUpdatePos)+"], child pos: [" + DhSectionPos.toString(parentUpdatePos) + "], Error: [" + e.getMessage() + "].", e);
}
finally
{
childReadLock.unlock();
this.lockedPosSet.remove(childPos);
}
}
if (DhSectionPos.getDetailLevel(parentUpdatePos) < TOP_SECTION_DETAIL_LEVEL)
{
parentDataSource.applyToParent = true;
}
this.updateDataSourceAtPos(parentUpdatePos, parentDataSource, false);
for (Long childPos : updatePosByParentPos.get(parentUpdatePos))
{
this.repo.setApplyToParent(childPos, false);
}
}
}
}
}
finally
{
if (parentLocked)
{
parentWriteLock.unlock();
this.lockedPosSet.remove(parentUpdatePos);
}
this.updatingPosSet.remove(parentUpdatePos);
}
});
}
catch (RejectedExecutionException ignore)
{ /* the executor was shut down, it should be back up shortly and able to accept new jobs */ }
catch (Exception e)
{
this.updatingPosSet.remove(parentUpdatePos);
throw e;
}
}
}
}
/** stops if it finds any LOD data */
private void runChildUpdates(PriorityTaskPicker.Executor executor, DhBlockPos targetBlockPos)
{
int maxUpdateTaskCount = getMaxUpdateTaskCount();
// queue child updates
if (executor.getQueueSize() < maxUpdateTaskCount
&& this.updatingPosSet.size() < maxUpdateTaskCount)
{
// get the positions that need to be applied to their children
LongArrayList childUpdatePosList = this.repo.getChildPositionsToUpdate(targetBlockPos.getX(), targetBlockPos.getZ(), maxUpdateTaskCount);
// queue the updates
for (long parentUpdatePos : childUpdatePosList)
{
// stop if there are already a bunch of updates queued
if (this.updatingPosSet.size() > maxUpdateTaskCount
|| executor.getQueueSize() > maxUpdateTaskCount)
{
break;
}
// skip already updating positions
if (!this.updatingPosSet.add(parentUpdatePos))
{
continue;
}
try
{
executor.execute(() ->
{
ReentrantLock parentReadLock = this.updateLockProvider.getLock(parentUpdatePos);
boolean parentLocked = false;
try
{
//LOGGER.info("updating parent: "+parentUpdatePos);
// Locking the parent before the children should prevent deadlocks.
// TryLock is used instead of lock so this thread can handle a different update.
if (parentReadLock.tryLock())
{
parentLocked = true;
this.lockedPosSet.add(parentUpdatePos);
try (FullDataSourceV2 parentDataSource = this.get(parentUpdatePos))
{
// will return null if the file handler is shutting down
if (parentDataSource != null)
{
// apply parent to each child
for (int i = 0; i < 4; i++)
{
long childPos = DhSectionPos.getChildByIndex(parentUpdatePos, i);
ReentrantLock childWriteLock = this.updateLockProvider.getLock(childPos);
try
{
childWriteLock.lock();
this.lockedPosSet.add(childPos);
try (FullDataSourceV2 childDataSource = this.get(childPos))
{
// will return null if the file handler is shutting down
if (childDataSource != null)
{
childDataSource.update(parentDataSource);
// don't propagate child updates past the bottom of the tree
if (DhSectionPos.getDetailLevel(childPos) != DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL)
{
childDataSource.applyToChildren = true;
}
this.updateDataSourceAtPos(childPos, childDataSource, false);
}
}
}
catch (Exception e)
{
LOGGER.error("Unexpected in child update propagation for parent pos: ["+DhSectionPos.toString(parentUpdatePos)+"], child pos: [" + DhSectionPos.toString(parentUpdatePos) + "], Error: [" + e.getMessage() + "].", e);
}
finally
{
childWriteLock.unlock();
this.lockedPosSet.remove(childPos);
}
}
this.repo.setApplyToChild(parentUpdatePos, false);
}
}
}
}
finally
{
if (parentLocked)
{
parentReadLock.unlock();
this.lockedPosSet.remove(parentUpdatePos);
}
this.updatingPosSet.remove(parentUpdatePos);
}
});
}
catch (RejectedExecutionException ignore)
{ /* the executor was shut down, it should be back up shortly and able to accept new jobs */ }
catch (Exception e)
{
this.updatingPosSet.remove(parentUpdatePos);
throw e;
}
}
}
}
@@ -645,7 +792,7 @@ public class FullDataSourceProviderV2
this.queuedUpdateCountsByPos
.forEach((pos, updateCountRef) -> { renderer.renderBox(new DebugRenderer.Box(pos, -32f, 80f + (updateCountRef.get() * 16f), 0.20f, Color.WHITE)); });
this.parentUpdatingPosSet
this.updatingPosSet
.forEach((pos) -> { renderer.renderBox(new DebugRenderer.Box(pos, -32f, 80f, 0.20f, Color.MAGENTA)); });
}
@@ -34,7 +34,6 @@ import com.seibel.distanthorizons.core.pooling.PhantomArrayListCheckout;
import com.seibel.distanthorizons.core.pooling.PhantomArrayListPool;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import com.seibel.distanthorizons.core.render.renderer.DebugRenderer;
import com.seibel.distanthorizons.core.render.renderer.IDebugRenderable;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker;
@@ -88,8 +87,20 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
// event listeners //
//=================//
public void addWorldGenCompleteListener(IOnWorldGenCompleteListener listener) { this.onWorldGenTaskCompleteListeners.add(listener); }
public void removeWorldGenCompleteListener(IOnWorldGenCompleteListener listener) { this.onWorldGenTaskCompleteListeners.remove(listener); }
public void addWorldGenCompleteListener(IOnWorldGenCompleteListener listener)
{
synchronized (this.onWorldGenTaskCompleteListeners)
{
this.onWorldGenTaskCompleteListeners.add(listener);
}
}
public void removeWorldGenCompleteListener(IOnWorldGenCompleteListener listener)
{
synchronized (this.onWorldGenTaskCompleteListeners)
{
this.onWorldGenTaskCompleteListeners.remove(listener);
}
}
@@ -129,10 +140,14 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
// TODO only fire after the section has finished generated or once every X seconds
private void fireOnGenPosSuccessListeners(long pos)
{
// fire the event listeners
for (IOnWorldGenCompleteListener listener : this.onWorldGenTaskCompleteListeners)
// synchronized to prevent a rare issue where the world generator is being shut down while this listener is firing
synchronized (this.onWorldGenTaskCompleteListeners)
{
listener.onWorldGenTaskComplete(pos);
// fire the event listeners
for (IOnWorldGenCompleteListener listener : this.onWorldGenTaskCompleteListeners)
{
listener.onWorldGenTaskComplete(pos);
}
}
}
@@ -195,16 +210,8 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
}
PriorityTaskPicker.Executor updateExecutor = ThreadPoolUtil.getUpdatePropagatorExecutor();
if (updateExecutor == null || updateExecutor.getQueueSize() >= MAX_UPDATE_TASK_COUNT / 2)
{
// don't queue additional world gen requests if the updater is behind
return false;
}
PriorityTaskPicker.Executor fileExecutor = ThreadPoolUtil.getFileHandlerExecutor();
if (fileExecutor == null || fileExecutor.getQueueSize() >= MAX_UPDATE_TASK_COUNT / 2)
if (fileExecutor == null || fileExecutor.getQueueSize() >= getMaxUpdateTaskCount() / 2)
{
// don't queue additional world gen requests if the file handler is overwhelmed,
// otherwise LODs may not load in properly
@@ -229,6 +236,11 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
// don't queue additional world gen requests if there are
// a lot of data sources in memory
// (this is done to prevent infinite memory growth)
// clear out the data sources that are in memory so
// we can start queuing new world gen tasks
this.delayedFullDataSourceSaveCache.flush();
return false;
}
@@ -298,7 +310,12 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
public boolean isFullyGenerated(ByteArrayList columnGenerationSteps)
{
return IntStream.range(0, columnGenerationSteps.size())
.noneMatch(i -> columnGenerationSteps.getByte(i) == EDhApiWorldGenerationStep.EMPTY.value);
.noneMatch(i ->
{
byte value = columnGenerationSteps.getByte(i);
return value == EDhApiWorldGenerationStep.EMPTY.value
|| value == EDhApiWorldGenerationStep.DOWN_SAMPLED.value;
});
}
public static final PhantomArrayListPool ARRAY_LIST_POOL = new PhantomArrayListPool("Generated Provider");
@@ -328,7 +345,8 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
// check if any positions are ungenerated
for (int i = 0; i < columnGenStepArray.size(); i++)
{
if (columnGenStepArray.getByte(i) == EDhApiWorldGenerationStep.EMPTY.value)
if (columnGenStepArray.getByte(i) == EDhApiWorldGenerationStep.EMPTY.value
|| columnGenStepArray.getByte(i) == EDhApiWorldGenerationStep.DOWN_SAMPLED.value)
{
positionFullyGenerated = false;
break;
@@ -393,7 +411,8 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
}
}
if (currentMinWorldGenStep == EDhApiWorldGenerationStep.EMPTY)
if (currentMinWorldGenStep == EDhApiWorldGenerationStep.EMPTY
|| currentMinWorldGenStep == EDhApiWorldGenerationStep.DOWN_SAMPLED)
{
// queue the task
break checkWorldGenLoop;
@@ -402,7 +421,8 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
}
}
if (currentMinWorldGenStep != EDhApiWorldGenerationStep.EMPTY)
if (currentMinWorldGenStep != EDhApiWorldGenerationStep.EMPTY
&& currentMinWorldGenStep != EDhApiWorldGenerationStep.DOWN_SAMPLED)
{
// no world gen needed for this position
return;
@@ -455,6 +475,23 @@ public class GeneratedFullDataSourceProvider extends FullDataSourceProviderV2 im
};
}
@Override
public CompletableFuture<Boolean> shouldGenerateSplitChild(long pos)
{
return GeneratedFullDataSourceProvider.this.getAsync(pos).thenApply(fullDataSource ->
{
//noinspection TryFinallyCanBeTryWithResources
try
{
return !GeneratedFullDataSourceProvider.this.isFullyGenerated(fullDataSource.columnGenerationSteps);
}
finally
{
fullDataSource.close();
}
});
}
}
private CompletableFuture<Void> onDataSourceSaveAsync(FullDataSourceV2 fullDataSource)
{
@@ -99,7 +99,10 @@ public class RemoteFullDataSourceProvider extends GeneratedFullDataSourceProvide
Long timestamp = this.getTimestampForPos(pos);
if (timestamp != null)
{
this.syncOnLoadRequestQueue.submitRequest(pos, timestamp, this::updateDataSource);
this.syncOnLoadRequestQueue.submitRequest(pos, timestamp, fullDataSource ->
{
this.updateDataSourceAsync(fullDataSource).whenComplete((result, throwable) -> fullDataSource.close());
});
}
return super.get(pos);
@@ -106,6 +106,19 @@ public class ClientOnlySaveStructure implements ISaveStructure
});
}
@Override
public File getPre23SaveFolder(ILevelWrapper levelWrapper)
{
// Allow API users to override the save folder
IDhApiSaveStructure saveStructureOverride = OverrideInjector.INSTANCE.get(IDhApiSaveStructure.class);
if (saveStructureOverride != null)
{
return this.getSaveFolder(levelWrapper);
}
return getSaveFolderByLevelId(levelWrapper.getDimensionType().getName());
}
//================//
@@ -34,5 +34,7 @@ public interface ISaveStructure extends AutoCloseable
*/
File getSaveFolder(ILevelWrapper levelWrapper);
File getPre23SaveFolder(ILevelWrapper levelWrapper);
}
@@ -75,6 +75,8 @@ public class LocalSaveStructure implements ISaveStructure
});
}
@Override
public File getPre23SaveFolder(ILevelWrapper levelWrapper) { return this.getSaveFolder(levelWrapper); }
//==================//
@@ -110,7 +110,7 @@ public class BatchGenerator implements IDhApiWorldGenerator
targetStep = EDhApiWorldGenerationStep.FEATURES;
break;
case INTERNAL_SERVER:
targetStep = EDhApiWorldGenerationStep.LIGHT; // TODO using something other than LIGHT would be good for clarity
targetStep = EDhApiWorldGenerationStep.LIGHT;
break;
}
@@ -328,7 +328,8 @@ public class DhLightingEngine
continue;
}
if (relNeighbourBlockPos.getY() < neighbourChunk.getMinNonEmptyHeight() || relNeighbourBlockPos.getY() > neighbourChunk.getExclusiveMaxBuildHeight())
if (relNeighbourBlockPos.getY() < neighbourChunk.getMinNonEmptyHeight()
|| relNeighbourBlockPos.getY() >= neighbourChunk.getExclusiveMaxBuildHeight())
{
// the light pos is outside the chunk's min/max height,
// this can happen if given a chunk that hasn't finished generating
@@ -9,6 +9,7 @@ import com.seibel.distanthorizons.core.file.fullDatafile.GeneratedFullDataSource
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import com.seibel.distanthorizons.core.util.FormatUtil;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.util.objects.RollingAverage;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IServerLevelWrapper;
@@ -167,17 +168,15 @@ public class PregenManager
this.generatedPercentage.update((double) this.nextSectionSpiralIndex.get() / this.sectionsToGenerate);
double chunksToGenerate = Math.ceil(Math.sqrt(this.sectionsToGenerate) / 2 * 4 * 10) / 10; // ceil to nearest 0.1
int chunkRatePerSecond = (int) (1000 / this.averageTaskCompletionIntervalMs.getAverage() * 4 * 4);
double etaMs = this.averageTaskCompletionIntervalMs.getAverage() * (this.sectionsToGenerate - this.nextSectionSpiralIndex.get());
return MessageFormat.format("Generated radius: {0,number,#.###} / {1,number,#.#} chunks ({2,number,#.###%}), ETA: {3}",
return MessageFormat.format("Generated radius: {0,number,#.###} / {1,number,#.#} chunks ({2} cps, {3,number,#.###%}), ETA: {4}",
this.generatedRadius.getValue(),
chunksToGenerate,
chunkRatePerSecond,
this.generatedPercentage.getValue(),
Duration.ofMillis((long) etaMs).toString()
.substring(2)
.replaceAll("(\\d[HMS])(?!$)", "$1 ")
.replaceAll("\\.\\d+", "")
.toLowerCase()
FormatUtil.formatEta(Duration.ofMillis((long) etaMs))
);
}
@@ -16,6 +16,7 @@ import org.apache.logging.log4j.Logger;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.*;
public class RemoteWorldRetrievalQueue extends AbstractFullDataNetworkRequestQueue implements IFullDataSourceRetrievalQueue, IDebugRenderable
@@ -47,7 +48,7 @@ public class RemoteWorldRetrievalQueue extends AbstractFullDataNetworkRequestQue
public void startAndSetTargetPos(DhBlockPos2D targetPos) { super.tick(targetPos); }
@Override
public byte lowestDataDetail() { return LodUtil.BLOCK_DETAIL_LEVEL + 12; } // TODO should be the same as what the server's update propgator can provide
public byte lowestDataDetail() { return LodUtil.BLOCK_DETAIL_LEVEL + 12; } // TODO should be the same as what the server's update propagator can provide
@Override
public byte highestDataDetail() { return LodUtil.BLOCK_DETAIL_LEVEL; }
@@ -56,7 +57,11 @@ public class RemoteWorldRetrievalQueue extends AbstractFullDataNetworkRequestQue
{
long generationStartMsTime = System.currentTimeMillis();
return super.submitRequest(sectionPos, tracker.getDataSourceConsumer())
return super.submitRequest(sectionPos, fullDataSource -> {
Objects.requireNonNull(tracker.getDataSourceConsumer()).accept(fullDataSource);
fullDataSource.close();
})
.thenApply(requestResult ->
{
long totalGenTimeInMs = System.currentTimeMillis() - generationStartMsTime;
@@ -74,7 +79,14 @@ public class RemoteWorldRetrievalQueue extends AbstractFullDataNetworkRequestQue
return WorldGenResult.CreateFail();
case REQUIRES_SPLITTING:
List<CompletableFuture<WorldGenResult>> childFutures = new ArrayList<>(4);
DhSectionPos.forEachChild(sectionPos, childPos -> childFutures.add(this.submitRetrievalTask(childPos, requiredDataDetail, tracker)));
DhSectionPos.forEachChild(sectionPos, childPos -> {
tracker.shouldGenerateSplitChild(childPos).thenAccept(shouldGenerate -> {
if (shouldGenerate)
{
childFutures.add(this.submitRetrievalTask(childPos, requiredDataDetail, tracker));
}
});
});
return WorldGenResult.CreateSplit(childFutures);
}
@@ -111,6 +123,18 @@ public class RemoteWorldRetrievalQueue extends AbstractFullDataNetworkRequestQue
return DhSectionPos.getChebyshevSignedBlockDistance(sectionPos, targetPos) <= this.networkState.sessionConfig.getMaxGenerationRequestDistance() * 16;
}
@Override
protected boolean onBeforeRequest(long sectionPos, CompletableFuture<ERequestResult> future)
{
if (DhSectionPos.getDetailLevel(sectionPos) > DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL
&& !Config.Server.Experimental.enableNSizedGeneration.get())
{
future.complete(ERequestResult.REQUIRES_SPLITTING);
return false;
}
return true;
}
@Override
protected String getQueueName() { return "World Remote Generation Queue"; }
@@ -26,6 +26,7 @@ import com.seibel.distanthorizons.api.objects.data.DhApiChunk;
import com.seibel.distanthorizons.api.objects.data.IDhApiFullDataSource;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.file.AbstractDataSourceHandler;
import com.seibel.distanthorizons.core.generation.tasks.IWorldGenTaskTracker;
import com.seibel.distanthorizons.core.generation.tasks.InProgressWorldGenTaskGroup;
import com.seibel.distanthorizons.core.generation.tasks.WorldGenResult;
@@ -103,7 +104,7 @@ public class WorldGenerationQueue implements IFullDataSourceRetrievalQueue, IDeb
private int estimatedRemainingTaskCount = 0;
private int estimatedRemainingChunkCount = 0;
private final RollingAverage rollingAverageChunkGenTimeInMs = new RollingAverage(1_000);
private final RollingAverage rollingAverageChunkGenTimeInMs = new RollingAverage(Runtime.getRuntime().availableProcessors() * 500);
public RollingAverage getRollingAverageChunkGenTimeInMs() { return this.rollingAverageChunkGenTimeInMs; }
@@ -439,9 +440,8 @@ public class WorldGenerationQueue implements IFullDataSourceRetrievalQueue, IDeb
ThreadPoolUtil.getWorldGenExecutor(),
(DhApiChunk dataPoints) ->
{
try
try(FullDataSourceV2 dataSource = LodDataBuilder.createFromApiChunkData(dataPoints, this.generator.runApiValidation()))
{
FullDataSourceV2 dataSource = LodDataBuilder.createFromApiChunkData(dataPoints, this.generator.runApiValidation());
dataSourceConsumer.accept(dataSource);
}
catch (DataCorruptedException | IllegalArgumentException e)
@@ -464,6 +464,11 @@ public class WorldGenerationQueue implements IFullDataSourceRetrievalQueue, IDeb
// set here so the API user doesn't have to pass in this value anywhere themselves
pooledDataSource.setRunApiChunkValidation(this.generator.runApiValidation());
// only apply to children if we aren't at the bottom of the tree
pooledDataSource.applyToChildren = DhSectionPos.getDetailLevel(pooledDataSource.getPos()) > DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL;
pooledDataSource.applyToParent = DhSectionPos.getDetailLevel(pooledDataSource.getPos()) < DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL + 12;
return this.generator.generateLod(
chunkPosMin.getX(), chunkPosMin.getZ(),
DhSectionPos.getX(requestPos), DhSectionPos.getZ(requestPos),
@@ -471,11 +476,19 @@ public class WorldGenerationQueue implements IFullDataSourceRetrievalQueue, IDeb
pooledDataSource,
generatorMode,
ThreadPoolUtil.getWorldGenExecutor(),
(IDhApiFullDataSource dataSource) ->
(IDhApiFullDataSource apiDataSource) ->
{
try
{
dataSourceConsumer.accept((FullDataSourceV2)dataSource);
FullDataSourceV2 fullDataSource = (FullDataSourceV2) apiDataSource;
try
{
dataSourceConsumer.accept(fullDataSource);
}
finally
{
fullDataSource.close();
}
}
catch (IllegalArgumentException e)
{
@@ -22,6 +22,7 @@ package com.seibel.distanthorizons.core.generation.tasks;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import javax.annotation.Nullable;
import java.util.concurrent.CompletableFuture;
import java.util.function.Consumer;
/**
@@ -33,4 +34,6 @@ public interface IWorldGenTaskTracker
@Nullable
Consumer<FullDataSourceV2> getDataSourceConsumer();
CompletableFuture<Boolean> shouldGenerateSplitChild(long pos);
}
@@ -28,6 +28,7 @@ import com.seibel.distanthorizons.core.jar.installer.GitlabGetter;
import com.seibel.distanthorizons.core.jar.installer.ModrinthGetter;
import com.seibel.distanthorizons.core.jar.installer.WebDownloader;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.logging.f3.F3Screen;
import com.seibel.distanthorizons.core.wrapperInterfaces.IVersionConstants;
import com.seibel.distanthorizons.coreapi.ModInfo;
import com.seibel.distanthorizons.coreapi.util.StringUtil;
@@ -42,8 +43,12 @@ import java.net.URLEncoder;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.MessageDigest;
import java.text.NumberFormat;
import java.util.Collections;
import java.util.Comparator;
import java.util.zip.CRC32;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipFile;
/**
* Used to update the mod automatically
@@ -253,94 +258,177 @@ public class SelfUpdater
deleteOldJarOnJvmShutdown = true;
LOGGER.info("Distant Horizons successfully updated. It will apply on game's relaunch");
String message = "Distant Horizons successfully updated. It will apply on game's relaunch";
LOGGER.info(message);
new Thread(() ->
{
String message = "Distant Horizons updated, this will be applied on game restart.";
if (!GraphicsEnvironment.isHeadless())
try
{
TinyFileDialogs.tinyfd_messageBox(ModInfo.READABLE_NAME, message, "ok", "info", false);
}
else
{
LOGGER.info(message);
}
catch (Exception ignore) { }
}).start();
return true;
}
catch (Exception e)
{
LOGGER.warn("Failed to update Distant Horizons to version [" + ModrinthGetter.getLatestNameForVersion(minecraftVersion) + "], error: ["+e.getMessage()+"].", e);
// delete the update file to prevent issues with a corrupt jar floating around
try
{
Files.deleteIfExists(file.toPath());
}
catch (Exception deleteCorruptFileException)
{
LOGGER.error("Unable to delete corrupted update file at ["+file.toPath()+"], error: ["+deleteCorruptFileException.getMessage()+"].", deleteCorruptFileException);
}
String message = "Failed to update Distant Horizons to version [" + ModrinthGetter.getLatestNameForVersion(minecraftVersion) + "], error: ["+e.getMessage()+"].";
LOGGER.error(message, e);
try
{
TinyFileDialogs.tinyfd_messageBox(ModInfo.READABLE_NAME, message, "ok", "error", false);
}
catch (Exception ignore) { }
return false;
}
}
public static boolean updateNightlyMod(String minecraftVersion, File file)
{
if (GitlabGetter.INSTANCE.projectPipelines.size() == 0)
if (GitlabGetter.INSTANCE.projectPipelines.isEmpty())
{
LOGGER.warn("Failed to find any nightly builds for the minecraft version ["+minecraftVersion+"] update canceled.");
return false;
}
Path mergedZipPath = null;
try
{
LOGGER.info("Attempting to auto update Distant Horizons.");
Files.createDirectories(file.getParentFile().toPath());
File mergedZip = file.getParentFile().toPath().resolve("merged.zip").toFile();
mergedZipPath = file.getParentFile().toPath().resolve("merged.zip");
WebDownloader.downloadAsFile(GitlabGetter.INSTANCE.getDownloads(GitlabGetter.INSTANCE.projectPipelines.get(0).get("id")).get(minecraftVersion), mergedZipPath.toFile());
WebDownloader.downloadAsFile(GitlabGetter.INSTANCE.getDownloads(GitlabGetter.INSTANCE.projectPipelines.get(0).get("id")).get(minecraftVersion), mergedZip);
ZipInputStream zis = new ZipInputStream(new FileInputStream(mergedZip));
ZipEntry zipEntry = zis.getNextEntry();
while (zipEntry != null)
try (ZipFile zipFile = new ZipFile(mergedZipPath.toFile()))
{
if (!zipEntry.isDirectory() && zipEntry.getName().contains("Merged")) // Look until the merged jar is found
ZipEntry zipEntry =
Collections.list(zipFile.entries()).stream()
.max(Comparator.comparingInt(entry -> entry.getName().length()))
// shouldn't happen, but just in case
.orElseThrow(() -> new Exception("Unable to find jar in zip. Is the downloaded zip empty?"));
// expected values as defined by the zip
long expectedCheckSum = zipEntry.getCrc();
int expectedSize = (int)zipEntry.getSize();
// read in the file content
byte[] buffer = new byte[expectedSize];
CRC32 crcCheckSumGenerator = new CRC32();
InputStream inputStream = zipFile.getInputStream(zipEntry);
int byteReadIndex = 0;
try
{
// write file content
FileOutputStream fos = new FileOutputStream(file);
byte[] buffer = new byte[1024];
NumberFormat outputFormat = NumberFormat.getNumberInstance();
int len;
while ((len = zis.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
fos.close();
deleteOldJarOnJvmShutdown = true;
LOGGER.info("Distant Horizons successfully updated. It will apply on game's relaunch");
new Thread(() ->
int nextByte = inputStream.read();
while (nextByte != -1)
{
String message = "Distant Horizons updated, this will be applied on game restart.";
if (!GraphicsEnvironment.isHeadless())
buffer[byteReadIndex] = (byte) nextByte;
crcCheckSumGenerator.update(nextByte);
nextByte = inputStream.read();
byteReadIndex++;
// TODO it would be better to change this divisor based on the expected size,
// so it would always be split up into 100 1% increments
// but this will work for now when the expected size is about 17 MB, this will log about 170 times
if (byteReadIndex % 100_000 == 0)
{
TinyFileDialogs.tinyfd_messageBox(ModInfo.READABLE_NAME, message, "ok", "info", false);
LOGGER.info("Decompressing ["+outputFormat.format(((double)byteReadIndex / expectedSize)*100.0)+"]%");
}
else
{
LOGGER.info(message);
}
}).start();
zis.close();
Files.deleteIfExists(newFileLocation.getParentFile().toPath().resolve("merged.zip"));
return true;
}
}
catch (EOFException ignore) { /* shouldn't happen, but just in case */ }
// confirm we read the whole file
if (byteReadIndex != expectedSize) // +1 on the index isn't necessary since the readIndex will always end +1 from where it started
{
LOGGER.warn("Distant Horizons update decompression failed, aborting install");
throw new Exception("Decompression failed");
}
zipEntry = zis.getNextEntry();
// confirm the checksum is correct (IE we decompressed correctly)
long actualChecksum = crcCheckSumGenerator.getValue();
if (actualChecksum != expectedCheckSum)
{
LOGGER.warn("Distant Horizons checksum mismatch, aborting install");
throw new Exception("Checksum Mismatch");
}
Files.write(file.toPath(), buffer);
}
zis.close();
return false;
Files.deleteIfExists(mergedZipPath);
deleteOldJarOnJvmShutdown = true;
String message = "Distant Horizons updated, this will be applied on game restart.";
LOGGER.info(message);
new Thread(() ->
{
try
{
TinyFileDialogs.tinyfd_messageBox(ModInfo.READABLE_NAME, message, "ok", "info", false);
}
catch (Exception ignore) { }
}).start();
return true;
}
catch (Exception e)
{
LOGGER.warn("Failed to update [" + ModInfo.READABLE_NAME + "] to version [" + GitlabGetter.INSTANCE.projectPipelines.get(0).get("sha") + "].", e);
// delete the update jar to prevent issues with a corrupt jar floating around
try
{
Files.deleteIfExists(file.toPath());
}
catch (Exception deleteCorruptFileException)
{
LOGGER.error("Unable to delete corrupted update jar file at ["+file.toPath()+"], error: ["+deleteCorruptFileException.getMessage()+"].", deleteCorruptFileException);
}
// delete the update zip so we can clean up
try
{
if (mergedZipPath != null)
{
Files.deleteIfExists(mergedZipPath);
}
}
catch (Exception deleteCorruptFileException)
{
LOGGER.error("Unable to delete corrupted update zip file at ["+mergedZipPath+"], error: ["+deleteCorruptFileException.getMessage()+"].", deleteCorruptFileException);
}
String message = "Failed to update [" + ModInfo.READABLE_NAME + "] to version [" + GitlabGetter.INSTANCE.projectPipelines.get(0).get("sha") + "], error: ["+e.getMessage()+"].";
LOGGER.error(message, e);
try
{
TinyFileDialogs.tinyfd_messageBox(ModInfo.READABLE_NAME, message, "ok", "error", false);
}
catch (Exception ignore) { }
return false;
}
}
@@ -21,12 +21,12 @@ package com.seibel.distanthorizons.core.level;
import com.seibel.distanthorizons.api.methods.events.abstractEvents.DhApiChunkModifiedEvent;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.file.beacon.BeaconBeamDataHandler;
import com.seibel.distanthorizons.core.file.fullDatafile.DelayedFullDataSourceSaveCache;
import com.seibel.distanthorizons.core.generation.DhLightingEngine;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.pos.DhChunkPos;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos;
import com.seibel.distanthorizons.core.render.renderer.generic.CloudRenderHandler;
import com.seibel.distanthorizons.core.render.renderer.generic.GenericObjectRenderer;
import com.seibel.distanthorizons.core.sql.dto.BeaconBeamDTO;
@@ -34,6 +34,7 @@ import com.seibel.distanthorizons.core.sql.dto.ChunkHashDTO;
import com.seibel.distanthorizons.core.sql.repo.AbstractDhRepo;
import com.seibel.distanthorizons.core.sql.repo.BeaconBeamRepo;
import com.seibel.distanthorizons.core.sql.repo.ChunkHashRepo;
import com.seibel.distanthorizons.core.util.KeyedLockContainer;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.wrapperInterfaces.chunk.IChunkWrapper;
import com.seibel.distanthorizons.coreapi.DependencyInjection.ApiEventInjector;
@@ -43,10 +44,12 @@ import org.jetbrains.annotations.Nullable;
import java.io.File;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.ReentrantLock;
public abstract class AbstractDhLevel implements IDhLevel
{
@@ -59,6 +62,8 @@ public abstract class AbstractDhLevel implements IDhLevel
@Nullable
public BeaconBeamRepo beaconBeamRepo;
protected final KeyedLockContainer<Long> beaconUpdateLockContainer = new KeyedLockContainer<>();
protected final DelayedFullDataSourceSaveCache delayedFullDataSourceSaveCache = new DelayedFullDataSourceSaveCache(this::onDataSourceSaveAsync, 3_000);
/** contains the {@link DhChunkPos} for each {@link DhSectionPos} that are queued to save */
protected final ConcurrentHashMap<Long, HashSet<DhChunkPos>> updatedChunkPosSetBySectionPos = new ConcurrentHashMap<>();
@@ -67,7 +72,6 @@ public abstract class AbstractDhLevel implements IDhLevel
/** Will be null if clouds shouldn't be rendered for this level. */
@Nullable
protected CloudRenderHandler cloudRenderHandler;
protected BeaconBeamDataHandler beaconBeamDataHandler;
@@ -126,13 +130,6 @@ public abstract class AbstractDhLevel implements IDhLevel
}
}
}
// shouldn't happen, but just in case
if (this.beaconBeamRepo != null)
{
this.beaconBeamDataHandler = new BeaconBeamDataHandler(this.beaconBeamRepo, genericRenderer);
}
}
@@ -228,31 +225,139 @@ public abstract class AbstractDhLevel implements IDhLevel
//=================//
@Override
public void updateBeaconBeamsForChunk(IChunkWrapper chunkToUpdate, ArrayList<IChunkWrapper> nearbyChunkList)
public void updateBeaconBeamsForSectionPos(long sectionPos, List<BeaconBeamDTO> activeBeamList)
{
if (this.beaconBeamDataHandler != null)
int minBlockX = DhSectionPos.getMinCornerBlockX(sectionPos);
int minBlockZ = DhSectionPos.getMinCornerBlockZ(sectionPos);
// TODO special logic had to be done for DhChunkPos.getMaxBlock,
// does that need to be done here?
// The DhChunkPos issue caused beacons to appear/disappear incorrectly on negative chunk borders
int maxBlockX = minBlockX + DhSectionPos.getBlockWidth(sectionPos);
int maxBlockZ = minBlockZ + DhSectionPos.getBlockWidth(sectionPos);
this.updateBeaconBeamsBetweenBlockPos(
sectionPos,
minBlockX, maxBlockX,
minBlockZ, maxBlockZ,
activeBeamList
);
}
@Override
public void updateBeaconBeamsForChunkPos(DhChunkPos chunkPos, List<BeaconBeamDTO> activeBeamList)
{
long sectionPos = DhSectionPos.encodeContaining(DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL, chunkPos);
int minBlockX = chunkPos.getMinBlockX();
int minBlockZ = chunkPos.getMinBlockZ();
int maxBlockX = chunkPos.getMaxBlockX();
int maxBlockZ = chunkPos.getMaxBlockZ();
//LOGGER.info("beacons ["+activeBeamList.size()+"] at ["+chunkPos+"] x["+minBlockX+"]-["+maxBlockX+"] z["+minBlockZ+"]-["+maxBlockZ+"].");
this.updateBeaconBeamsBetweenBlockPos(
sectionPos,
minBlockX, maxBlockX,
minBlockZ, maxBlockZ,
activeBeamList
);
}
private void updateBeaconBeamsBetweenBlockPos(
long sectionPosForLock,
int minBlockX, int maxBlockX,
int minBlockZ, int maxBlockZ,
List<BeaconBeamDTO> activeBeamList
) // TODO min/max block pos instead
{
if (this.beaconBeamRepo == null)
{
List<BeaconBeamDTO> activeBeamList = chunkToUpdate.getAllActiveBeacons(nearbyChunkList);
this.beaconBeamDataHandler.setBeaconBeamsForChunk(chunkToUpdate.getChunkPos(), activeBeamList);
return;
}
// locked to prevent two threads from updating the same section at the same time
ReentrantLock lock = this.beaconUpdateLockContainer.getLockForPos(sectionPosForLock); // TODO this can cause a lot of slow-downs
try
{
lock.lock();
HashSet<DhBlockPos> allPosSet = new HashSet<>();
// sort new beams
HashMap<DhBlockPos, BeaconBeamDTO> activeBeamByPos = new HashMap<>(activeBeamList.size());
for (BeaconBeamDTO beam : activeBeamList)
{
activeBeamByPos.put(beam.blockPos, beam);
allPosSet.add(beam.blockPos);
}
// get existing beams
List<BeaconBeamDTO> existingBeamList = this.beaconBeamRepo.getAllBeamsInBlockPosRange(
minBlockX, maxBlockX,
minBlockZ, maxBlockZ);
HashMap<DhBlockPos, BeaconBeamDTO> existingBeamByPos = new HashMap<>(existingBeamList.size());
for (BeaconBeamDTO beam : existingBeamList)
{
existingBeamByPos.put(beam.blockPos, beam);
allPosSet.add(beam.blockPos);
}
for (DhBlockPos beaconPos : allPosSet)
{
if (minBlockX <= beaconPos.getX() && beaconPos.getX() <= maxBlockX
&& minBlockZ <= beaconPos.getZ() && beaconPos.getZ() <= maxBlockZ)
{
//// don't modify beacons outside the updated range
//continue;
}
else
{
continue;
}
BeaconBeamDTO existingBeam = existingBeamByPos.get(beaconPos);
BeaconBeamDTO activeBeam = activeBeamByPos.get(beaconPos);
if (activeBeam != null)
{
//LOGGER.info("add beacon ["+activeBeam.blockPos+"] x["+minBlockX+"]-["+maxBlockX+"] z["+minBlockZ+"]-["+maxBlockZ+"].");
if (existingBeam == null)
{
// new beam found, add to DB
this.beaconBeamRepo.save(activeBeam);
}
else
{
// beam still exists in chunk
if (!existingBeam.color.equals(activeBeam.color))
{
// beam colors were changed
this.beaconBeamRepo.save(activeBeam);
}
}
}
else if (existingBeam != null)
{
// beam no longer exists at position, remove from DB
this.beaconBeamRepo.deleteWithKey(beaconPos);
//LOGGER.info("remove beacon ["+beaconPos+"] x["+minBlockX+"]-["+maxBlockX+"] z["+minBlockZ+"]-["+maxBlockZ+"].");
}
}
}
finally
{
lock.unlock();
}
}
@Override
public void loadBeaconBeamsInPos(long pos)
{
if (this.beaconBeamDataHandler != null)
{
this.beaconBeamDataHandler.loadBeaconBeamsInPos(pos);
}
}
@Override
public void unloadBeaconBeamsInPos(long pos)
{
if (this.beaconBeamDataHandler != null)
{
this.beaconBeamDataHandler.unloadBeaconBeamsInPos(pos);
}
}
@Nullable
public BeaconBeamRepo getBeaconBeamRepo() { return this.beaconBeamRepo; }
@@ -23,7 +23,6 @@ import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.util.math.Vec3d;
import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
import com.seibel.distanthorizons.core.wrapperInterfaces.misc.IServerPlayerWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.ILevelWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IServerLevelWrapper;
@@ -31,7 +30,6 @@ import org.apache.logging.log4j.Logger;
import javax.annotation.Nullable;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.*;
public abstract class AbstractDhServerLevel extends AbstractDhLevel implements IDhServerLevel
@@ -154,7 +152,8 @@ public abstract class AbstractDhServerLevel extends AbstractDhLevel implements I
if (Config.Server.generationBoundsRadius.get() > 0)
{
if (DhSectionPos.getChebyshevSignedBlockDistance(message.sectionPos, new DhBlockPos2D(
Config.Server.generationBoundsX.get(), Config.Server.generationBoundsZ.get()
serverPlayerState.sessionConfig.getGenerationBoundsX(),
serverPlayerState.sessionConfig.getGenerationBoundsZ()
)) > Config.Server.generationBoundsRadius.get())
{
message.sendResponse(new RequestOutOfRangeException("Section out of allowed bounds"));
@@ -162,7 +161,7 @@ public abstract class AbstractDhServerLevel extends AbstractDhLevel implements I
}
}
if (Config.Server.generateOnlyInHighestDetail.get() && DhSectionPos.getDetailLevel(message.sectionPos) != DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL)
if (!Config.Server.Experimental.enableNSizedGeneration.get() && DhSectionPos.getDetailLevel(message.sectionPos) != DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL)
{
message.sendResponse(new SectionRequiresSplittingException("Only highest-detail sections are allowed"));
return;
@@ -278,8 +277,7 @@ public abstract class AbstractDhServerLevel extends AbstractDhLevel implements I
Vec3d playerPosition = serverPlayerState.getServerPlayer().getPosition();
int distanceFromPlayer = DhSectionPos.getChebyshevSignedBlockDistance(data.getPos(), new DhBlockPos2D((int) playerPosition.x, (int) playerPosition.z)) / 16;
if (distanceFromPlayer >= serverPlayerState.getServerPlayer().getViewDistance()
&& distanceFromPlayer <= serverPlayerState.sessionConfig.getMaxUpdateDistanceRadius())
if (distanceFromPlayer <= serverPlayerState.sessionConfig.getMaxUpdateDistanceRadius())
{
serverPlayerState.fullDataPayloadSender.sendInChunks(payload, () ->
{
@@ -258,7 +258,11 @@ public class ClientLevelModule implements Closeable, AbstractDataSourceHandler.I
public void clearRenderCache()
{
this.clientLevel.getClientLevelWrapper().clearBlockColorCache();
IClientLevelWrapper clientLevelWrapper = this.clientLevel.getClientLevelWrapper();
if (clientLevelWrapper != null)
{
clientLevelWrapper.clearBlockColorCache();
}
ClientRenderState ClientRenderState = this.ClientRenderStateRef.get();
if (ClientRenderState != null && ClientRenderState.quadtree != null)
@@ -19,46 +19,56 @@
package com.seibel.distanthorizons.core.level;
import com.google.common.cache.CacheBuilder;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiRenderParam;
import com.seibel.distanthorizons.core.config.AppliedConfigState;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.file.beacon.BeaconBeamDataHandler;
import com.seibel.distanthorizons.core.file.fullDatafile.FullDataSourceProviderV2;
import com.seibel.distanthorizons.core.file.fullDatafile.RemoteFullDataSourceProvider;
import com.seibel.distanthorizons.core.file.structure.ISaveStructure;
import com.seibel.distanthorizons.core.generation.RemoteWorldRetrievalQueue;
import com.seibel.distanthorizons.core.logging.ConfigBasedLogger;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.multiplayer.client.ClientNetworkState;
import com.seibel.distanthorizons.core.multiplayer.client.SyncOnLoadRequestQueue;
import com.seibel.distanthorizons.core.network.event.ScopedNetworkEventSource;
import com.seibel.distanthorizons.core.network.messages.fullData.FullDataPartialUpdateMessage;
import com.seibel.distanthorizons.core.pos.DhChunkPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos;
import com.seibel.distanthorizons.core.render.RenderBufferHandler;
import com.seibel.distanthorizons.core.render.renderer.generic.GenericObjectRenderer;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import com.seibel.distanthorizons.core.render.renderer.DebugRenderer;
import com.seibel.distanthorizons.core.sql.dto.FullDataSourceV2DTO;
import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
import com.seibel.distanthorizons.core.wrapperInterfaces.block.IBlockStateWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftClientWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IProfilerWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IBiomeWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IClientLevelWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.ILevelWrapper;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.jetbrains.annotations.Nullable;
import javax.annotation.CheckForNull;
import java.awt.*;
import java.io.File;
import java.util.*;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;
import java.util.concurrent.TimeUnit;
/** The level used when connected to a server */
public class DhClientLevel extends AbstractDhLevel implements IDhClientLevel
{
private static final Logger LOGGER = DhLoggerBuilder.getLogger();
protected static final ConfigBasedLogger NETWORK_LOGGER = new ConfigBasedLogger(LogManager.getLogger(),
() -> Config.Common.Logging.logNetworkEvent.get());
private static final IMinecraftClientWrapper MC_CLIENT = SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class);
public final ClientLevelModule clientside;
@@ -71,6 +81,13 @@ public class DhClientLevel extends AbstractDhLevel implements IDhClientLevel
@Nullable
private final ScopedNetworkEventSource networkEventSource;
private final Set<DhChunkPos> loadedOnceChunks = Collections.newSetFromMap(
CacheBuilder.newBuilder()
.expireAfterWrite(10, TimeUnit.MINUTES)
.<DhChunkPos, Boolean>build()
.asMap()
);
public final WorldGenModule worldGenModule;
public final AppliedConfigState<Boolean> worldGeneratorEnabledConfig;
@@ -87,10 +104,21 @@ public class DhClientLevel extends AbstractDhLevel implements IDhClientLevel
{ this(saveStructure, clientLevelWrapper, null, true, networkState); }
public DhClientLevel(ISaveStructure saveStructure, IClientLevelWrapper clientLevelWrapper, @Nullable File fullDataSaveDirOverride, boolean enableRendering, @Nullable ClientNetworkState networkState)
{
if (saveStructure.getSaveFolder(clientLevelWrapper).mkdirs())
File saveFolder = saveStructure.getSaveFolder(clientLevelWrapper);
File pre23Folder = saveStructure.getPre23SaveFolder(clientLevelWrapper);
if (pre23Folder.exists())
{
if (!pre23Folder.renameTo(saveFolder))
{
throw new RuntimeException("Could not move old save data folder: " + pre23Folder.getAbsolutePath() + " to " + saveFolder.getAbsolutePath());
}
}
else if (saveStructure.getSaveFolder(clientLevelWrapper).mkdirs())
{
LOGGER.warn("unable to create data folder.");
}
this.levelWrapper = clientLevelWrapper;
this.levelWrapper.setParentLevel(this);
this.saveStructure = saveStructure;
@@ -135,18 +163,38 @@ public class DhClientLevel extends AbstractDhLevel implements IDhClientLevel
return;
}
try (FullDataSourceV2DTO dataSourceDto = this.networkState.fullDataPayloadReceiver.decodeDataSourceAndReleaseBuffer(message.payload))
try(FullDataSourceV2DTO dataSourceDto = this.networkState.fullDataPayloadReceiver.decodeDataSourceAndReleaseBuffer(message.payload))
{
if (!message.isSameLevelAs(this.levelWrapper))
boolean isSameLevel = message.isSameLevelAs(this.levelWrapper);
NETWORK_LOGGER.debug("Buffer {} isSameLevel: {}", message.payload.dtoBufferId, isSameLevel);
if (!isSameLevel)
{
return;
}
this.beaconBeamDataHandler.setBeaconBeamsForPos(dataSourceDto.pos, message.payload.beaconBeams);
try (FullDataSourceV2 fullDataSource = dataSourceDto.createDataSource(this.levelWrapper))
Executor executor = ThreadPoolUtil.getFileHandlerExecutor();
if (executor != null)
{
this.updateDataSourcesAsync(fullDataSource);
executor.execute(() ->
{
try
{
// TODO this has a lock which can cause stuttering/lag issues
this.updateBeaconBeamsForSectionPos(dataSourceDto.pos, message.payload.beaconBeams);
}
catch (Exception e)
{
LOGGER.error("Unexpected erorr while updating full data source, error: ["+e.getMessage()+"].", e);
}
});
}
FullDataSourceV2 fullDataSource = dataSourceDto.createDataSource(this.levelWrapper);
this.updateDataSourcesAsync(fullDataSource)
.whenComplete((result, e) -> fullDataSource.close());
}
catch (Exception e)
{
@@ -284,7 +332,16 @@ public class DhClientLevel extends AbstractDhLevel implements IDhClientLevel
ClientLevelModule.ClientRenderState renderState = this.clientside.ClientRenderStateRef.get();
return (renderState != null) ? renderState.renderBufferHandler : null;
}
public BeaconBeamDataHandler getBeaconBeamDataHandler() { return this.beaconBeamDataHandler; }
public boolean shouldProcessChunkUpdate(DhChunkPos chunkPos)
{
if (this.networkState == null || !this.networkState.isReady())
{
return true;
}
return !this.networkState.sessionConfig.isRealTimeUpdatesEnabled() || this.loadedOnceChunks.add(chunkPos);
}
@@ -35,6 +35,7 @@ import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftCli
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IBiomeWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IClientLevelWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IServerLevelWrapper;
import org.jetbrains.annotations.Nullable;
import java.awt.*;
import java.util.List;
@@ -107,6 +108,7 @@ public class DhClientServerLevel extends AbstractDhServerLevel implements IDhCli
}
}
@Nullable
@Override
public IClientLevelWrapper getClientLevelWrapper() { return MC_CLIENT.getWrappedClientLevel(); }
@@ -25,6 +25,7 @@ import com.seibel.distanthorizons.core.wrapperInterfaces.block.IBlockStateWrappe
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IProfilerWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IBiomeWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.IClientLevelWrapper;
import org.jetbrains.annotations.Nullable;
public interface IDhClientLevel extends IDhLevel
{
@@ -35,6 +36,7 @@ public interface IDhClientLevel extends IDhLevel
int computeBaseColor(DhBlockPos pos, IBiomeWrapper biome, IBlockStateWrapper block);
@Nullable
IClientLevelWrapper getClientLevelWrapper();
/**
@@ -26,6 +26,8 @@ import com.seibel.distanthorizons.core.file.structure.ISaveStructure;
import com.seibel.distanthorizons.core.pos.DhChunkPos;
import com.seibel.distanthorizons.core.render.RenderBufferHandler;
import com.seibel.distanthorizons.core.render.renderer.generic.GenericObjectRenderer;
import com.seibel.distanthorizons.core.sql.dto.BeaconBeamDTO;
import com.seibel.distanthorizons.core.sql.repo.BeaconBeamRepo;
import com.seibel.distanthorizons.core.wrapperInterfaces.chunk.IChunkWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.world.ILevelWrapper;
import org.jetbrains.annotations.Nullable;
@@ -50,9 +52,16 @@ public interface IDhLevel extends AutoCloseable, GeneratedFullDataSourceProvider
int getChunkHash(DhChunkPos pos);
void updateChunkAsync(IChunkWrapper chunk, int newChunkHash);
void loadBeaconBeamsInPos(long pos);
void updateBeaconBeamsForChunk(IChunkWrapper chunkToUpdate, ArrayList<IChunkWrapper> nearbyChunkList);
void unloadBeaconBeamsInPos(long pos);
default void updateBeaconBeamsForChunk(IChunkWrapper chunkToUpdate, ArrayList<IChunkWrapper> nearbyChunkList)
{
List<BeaconBeamDTO> activeBeamList = chunkToUpdate.getAllActiveBeacons(nearbyChunkList);
this.updateBeaconBeamsForChunkPos(chunkToUpdate.getChunkPos(), activeBeamList);
}
void updateBeaconBeamsForChunkPos(DhChunkPos chunkPos, List<BeaconBeamDTO> activeBeamList);
void updateBeaconBeamsForSectionPos(long sectionPos, List<BeaconBeamDTO> activeBeamList);
@Nullable
BeaconBeamRepo getBeaconBeamRepo();
FullDataSourceProviderV2 getFullDataProvider();
@@ -27,6 +27,7 @@ import com.seibel.distanthorizons.core.generation.IFullDataSourceRetrievalQueue;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.logging.f3.F3Screen;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import com.seibel.distanthorizons.core.util.FormatUtil;
import com.seibel.distanthorizons.core.util.ThreadUtil;
import com.seibel.distanthorizons.core.util.objects.RollingAverage;
import com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker;
@@ -35,6 +36,7 @@ import com.seibel.distanthorizons.core.world.DhApiWorldProxy;
import org.apache.logging.log4j.Logger;
import java.io.Closeable;
import java.time.Duration;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ThreadPoolExecutor;
@@ -220,6 +222,9 @@ public class WorldGenModule implements Closeable
/** Handles the {@link IFullDataSourceRetrievalQueue} and any other necessary world gen information. */
public static abstract class AbstractWorldGenState
{
/** static so we only send the disable message once per session */
private static long firstProgressMessageSentMs = 0;
public IFullDataSourceRetrievalQueue worldGenerationQueue;
private static final ThreadPoolExecutor PROGRESS_UPDATER_THREAD = ThreadUtil.makeSingleDaemonThreadPool("World Gen Progress Updater");
@@ -280,23 +285,40 @@ public class WorldGenModule implements Closeable
}
private void sendRetrievalProgress()
{
// format the remaining chunks
int remainingChunkCount = this.worldGenerationQueue.getRetrievalEstimatedRemainingChunkCount();
remainingChunkCount += this.worldGenerationQueue.getQueuedChunkCount();
String remainingChunkCountStr = F3Screen.NUMBER_FORMAT.format(remainingChunkCount);
String message = "DH Gen/Import: " + remainingChunkCountStr + " chunks";
String message = "DH is loading chunks. " + remainingChunkCountStr + " left.";
// show a message about how to disable progress logging if requested
int msToShowDisableInstructions = Config.Common.WorldGenerator.generationProgressDisableMessageDisplayTimeInSeconds.get() * 1_000;
if (msToShowDisableInstructions > 0)
{
long timeSinceFirstMessageInMs = (System.currentTimeMillis() - firstProgressMessageSentMs);
// always show this message for the first tick
if (firstProgressMessageSentMs == 0
// show this message if there is still time
|| timeSinceFirstMessageInMs < msToShowDisableInstructions)
{
// append to the current message
message += " This message can be hidden in the DH config.";
}
}
// add the remaining time estimate if available
double chunksPerSec = this.getEstimatedChunksPerSecond();
if (chunksPerSec > 0)
{
long estimatedRemainingTime = (long) (remainingChunkCount / chunksPerSec);
message += " Estimated Time: " + formatSeconds(estimatedRemainingTime);//+ " at " + F3Screen.NUMBER_FORMAT.format(chunksPerSec) + " chunks/sec";
message += " ETA: " + FormatUtil.formatEta(Duration.ofSeconds(estimatedRemainingTime));//+ " at " + F3Screen.NUMBER_FORMAT.format(chunksPerSec) + " chunks/sec";
}
// only log if there are chunks needing to be generated
if (remainingChunkCount != 0)
{
// determine where to log
EDhApiDistantGeneratorProgressDisplayLocation displayLocation = Config.Common.WorldGenerator.showGenerationProgress.get();
if (displayLocation == EDhApiDistantGeneratorProgressDisplayLocation.OVERLAY)
{
@@ -311,33 +333,14 @@ public class WorldGenModule implements Closeable
LOGGER.info(message);
}
// mark when the first message was sent
if (firstProgressMessageSentMs == 0)
{
firstProgressMessageSentMs = System.currentTimeMillis();
}
}
}
private static String formatSeconds(long totalSeconds)
{
long days = totalSeconds / (24 * 3600); // 24 hours in a day
long hours = (totalSeconds % (24 * 3600)) / 3600; // Hours
long minutes = (totalSeconds % 3600) / 60; // Minutes
long seconds = totalSeconds % 60; // Seconds
String timeString = "";
if (days > 0)
{
timeString += days+" ";
}
if (hours > 0)
{
timeString += hours+":";
}
if (minutes > 0)
{
timeString += String.format("%02d", minutes)+":";
}
timeString += String.format("%02d", seconds);
return timeString;
}
/** @return -1 if this method isn't supported or available */
public double getEstimatedChunksPerSecond()
@@ -1,6 +1,7 @@
package com.seibel.distanthorizons.core.multiplayer.client;
import com.google.common.base.Stopwatch;
import com.google.common.cache.CacheBuilder;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.config.types.ConfigEntry;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
@@ -71,6 +72,16 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
private final SupplierBasedRateLimiter<Void> rateLimiter = new SupplierBasedRateLimiter<>(this::getRequestRateLimit);
private final Set<Long> succeededPositions = Collections.newSetFromMap(CacheBuilder.newBuilder()
.expireAfterWrite(20, TimeUnit.MINUTES)
.<Long, Boolean>build()
.asMap());
private final Set<Long> requiresSplittingPositions = Collections.newSetFromMap(CacheBuilder.newBuilder()
.expireAfterWrite(20, TimeUnit.MINUTES)
.<Long, Boolean>build()
.asMap());
//=============//
@@ -96,6 +107,7 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
protected abstract int getRequestRateLimit();
protected abstract boolean isSectionAllowedToGenerate(long sectionPos, DhBlockPos2D targetPos);
protected abstract boolean onBeforeRequest(long sectionPos, CompletableFuture<ERequestResult> future);
protected abstract String getQueueName();
@@ -105,12 +117,22 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
// request submitting //
//====================//
public CompletableFuture<RequestResult> submitRequest(long sectionPos, Consumer<FullDataSourceV2> dataSourceConsumer)
public CompletableFuture<ERequestResult> submitRequest(long sectionPos, Consumer<FullDataSourceV2> dataSourceConsumer)
{ return this.submitRequest(sectionPos, null, dataSourceConsumer); }
public CompletableFuture<RequestResult> submitRequest(long sectionPos, @Nullable Long clientTimestamp, Consumer<FullDataSourceV2> dataSourceConsumer)
public CompletableFuture<ERequestResult> submitRequest(long sectionPos, @Nullable Long clientTimestamp, Consumer<FullDataSourceV2> dataSourceConsumer)
{
if (this.succeededPositions.contains(sectionPos))
{
return CompletableFuture.completedFuture(ERequestResult.FAILED);
}
if (this.requiresSplittingPositions.contains(sectionPos))
{
return CompletableFuture.completedFuture(ERequestResult.REQUIRES_SPLITTING);
}
AtomicBoolean added = new AtomicBoolean(false);
RequestQueueEntry entry = this.waitingTasksBySectionPos.compute(sectionPos, (k, existingQueueEntry) ->
RequestQueueEntry entry = this.waitingTasksBySectionPos.compute(sectionPos, (pos, existingQueueEntry) ->
{
if (existingQueueEntry != null)
{
@@ -126,8 +148,10 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
{
case SUCCEEDED:
this.finishedRequests.incrementAndGet();
this.succeededPositions.add(pos);
return;
case REQUIRES_SPLITTING:
this.requiresSplittingPositions.add(sectionPos);
return;
case FAILED:
this.failedRequests.incrementAndGet();
@@ -147,7 +171,7 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
if (!added.get())
{
return CompletableFuture.completedFuture(RequestResult.FAILED);
return CompletableFuture.completedFuture(ERequestResult.FAILED);
}
return entry.future;
@@ -204,6 +228,12 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
return;
}
if (!this.onBeforeRequest(sectionPos, entry.future))
{
this.pendingTasksSemaphore.release();
return;
}
Long offsetEntryTimestamp = entry.updateTimestamp != null
? entry.updateTimestamp + this.networkState.getServerTimeOffset()
: null;
@@ -228,6 +258,11 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
{
FullDataSourceV2DTO dataSourceDto = this.networkState.fullDataPayloadReceiver.decodeDataSourceAndReleaseBuffer(response.payload);
// set application flags based on the received detail level,
// this is needed so the data sources propagate correctly
dataSourceDto.applyToChildren = DhSectionPos.getDetailLevel(dataSourceDto.pos) > DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL;
dataSourceDto.applyToParent = DhSectionPos.getDetailLevel(dataSourceDto.pos) < DhSectionPos.SECTION_BLOCK_DETAIL_LEVEL + 12;
AbstractExecutorService executor = ThreadPoolUtil.getNetworkCompressionExecutor();
if (executor == null)
{
@@ -240,11 +275,10 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
{
try
{
this.level.getBeaconBeamDataHandler().setBeaconBeamsForPos(dataSourceDto.pos, response.payload.beaconBeams);
try (FullDataSourceV2 fullDataSource = dataSourceDto.createDataSource(this.level.getLevelWrapper()))
{
entry.dataSourceConsumer.accept(fullDataSource);
}
this.level.updateBeaconBeamsForSectionPos(dataSourceDto.pos, response.payload.beaconBeams);
FullDataSourceV2 fullDataSource = dataSourceDto.createDataSource(this.level.getLevelWrapper());
entry.dataSourceConsumer.accept(fullDataSource);
}
catch (Exception e)
{
@@ -263,7 +297,7 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
}
catch (SectionRequiresSplittingException ignored)
{
return entry.future.complete(RequestResult.REQUIRES_SPLITTING);
return entry.future.complete(ERequestResult.REQUIRES_SPLITTING);
}
catch (SessionClosedException | CancellationException ignored)
{
@@ -272,11 +306,11 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
catch (RequestRejectedException e)
{
LOGGER.info("Request rejected by the server: " + e.getMessage());
return entry.future.complete(RequestResult.FAILED);
return entry.future.complete(ERequestResult.FAILED);
}
catch (RateLimitedException e)
{
LOGGER.warn("Rate limited by server, re-queueing task [" + DhSectionPos.toString(sectionPos) + "]: " + e.getMessage());
LOGGER.info("Rate limited by server, re-queueing task [" + DhSectionPos.toString(sectionPos) + "]: " + e.getMessage());
// Skip all requests for 1 second
this.rateLimiter.acquireAll();
@@ -304,11 +338,11 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
}
else
{
return entry.future.complete(RequestResult.FAILED);
return entry.future.complete(ERequestResult.FAILED);
}
}
return entry.future.complete(RequestResult.SUCCEEDED);
return entry.future.complete(ERequestResult.SUCCEEDED);
});
}
@@ -421,7 +455,7 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
protected static class RequestQueueEntry
{
/** encapsulates the entire request, including client side queuing and the actual server request */
public final CompletableFuture<RequestResult> future = new CompletableFuture<>();
public final CompletableFuture<ERequestResult> future = new CompletableFuture<>();
public final Consumer<FullDataSourceV2> dataSourceConsumer;
/** will be null if we want to retrieve the LOD regardless of when it was last updated */
@Nullable
@@ -451,7 +485,7 @@ public abstract class AbstractFullDataNetworkRequestQueue implements IDebugRende
}
public enum RequestResult
public enum ERequestResult
{
SUCCEEDED,
REQUIRES_SPLITTING,
@@ -0,0 +1,71 @@
package com.seibel.distanthorizons.core.multiplayer.client;
import com.seibel.distanthorizons.core.network.messages.fullData.FullDataSplitMessage;
import java.util.concurrent.atomic.AtomicLong;
import java.util.function.BooleanSupplier;
import java.util.function.IntConsumer;
import java.util.function.IntSupplier;
public class ClientCongestionControl
{
private static final double ADDITIVE_INCREASE = 50000;
private static final long INTERVAL_MS = 1000;
private final Runnable rateUpdateHandler;
private final AtomicLong bytesReceived = new AtomicLong(0);
private double desiredRate;
private long lastAdjustTime;
public ClientCongestionControl(
Runnable rateUpdateHandler
)
{
this.rateUpdateHandler = rateUpdateHandler;
this.reset();
}
public void reset()
{
this.desiredRate = ADDITIVE_INCREASE;
this.lastAdjustTime = System.currentTimeMillis();
this.bytesReceived.set(0);
}
public void onPayloadReceived(FullDataSplitMessage message)
{
long now = System.currentTimeMillis();
if (now - this.lastAdjustTime >= INTERVAL_MS)
{
this.adjustRate(now);
}
this.bytesReceived.addAndGet(message.buffer.readableBytes());
}
private void adjustRate(long now)
{
double throughput = this.bytesReceived.getAndSet(0);
if (throughput >= this.desiredRate)
{
this.desiredRate += ADDITIVE_INCREASE;
}
else
{
this.desiredRate = Math.max(throughput - ADDITIVE_INCREASE / 2, 1000);
}
this.lastAdjustTime = now;
this.rateUpdateHandler.run();
}
public int getDesiredRate()
{
return (int) (this.desiredRate / 1000);
}
}
@@ -1,6 +1,7 @@
package com.seibel.distanthorizons.core.multiplayer.client;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.config.listeners.ConfigChangeListener;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.logging.ConfigBasedLogger;
import com.seibel.distanthorizons.core.multiplayer.config.SessionConfig;
@@ -56,6 +57,23 @@ public class ClientNetworkState implements Closeable
private long serverTimeOffset = 0;
public long getServerTimeOffset() { return this.serverTimeOffset; }
private final ClientCongestionControl congestionControl = new ClientCongestionControl(
() -> {
if (Config.Server.enableAdaptiveTransferSpeed.get())
{
this.sendConfigMessage(false);
}
}
);
private final ConfigChangeListener<Boolean> adaptiveTransferSpeedListener = new ConfigChangeListener<>(Config.Server.enableAdaptiveTransferSpeed, isEnabled -> {
if (isEnabled)
{
this.congestionControl.reset();
}
this.sendConfigMessage();
});
//=============//
@@ -116,6 +134,7 @@ public class ClientNetworkState implements Closeable
});
this.networkSession.registerHandler(FullDataSplitMessage.class, this.fullDataPayloadReceiver::receiveChunk);
this.networkSession.registerHandler(FullDataSplitMessage.class, this.congestionControl::onPayloadReceived);
}
}
@@ -127,10 +146,22 @@ public class ClientNetworkState implements Closeable
public void sendConfigMessage()
public void sendConfigMessage() { this.sendConfigMessage(true); }
public void sendConfigMessage(boolean blocking)
{
this.configReceived = false;
this.getSession().sendMessage(new SessionConfigMessage(new SessionConfig()));
SessionConfig sessionConfig = new SessionConfig();
if (Config.Server.enableAdaptiveTransferSpeed.get())
{
sessionConfig.constrainValue(Config.Server.maxDataTransferSpeed, this.congestionControl.getDesiredRate());
}
if (blocking)
{
this.configReceived = false;
}
this.getSession().sendMessage(new SessionConfigMessage(sessionConfig));
}
@@ -166,6 +197,7 @@ public class ClientNetworkState implements Closeable
public void close()
{
this.fullDataPayloadReceiver.close();
this.adaptiveTransferSpeedListener.close();
this.configAnyChangeListener.close();
this.networkSession.close();
}
@@ -6,6 +6,8 @@ import com.seibel.distanthorizons.core.level.DhClientLevel;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import java.util.concurrent.CompletableFuture;
/**
* This queue only handles LOD updates for
* LODs that were changed when the player wasn't online
@@ -37,6 +39,8 @@ public class SyncOnLoadRequestQueue extends AbstractFullDataNetworkRequestQueue
{
return DhSectionPos.getChebyshevSignedBlockDistance(sectionPos, targetPos) <= this.networkState.sessionConfig.getMaxSyncOnLoadDistance() * 16;
}
@Override
protected boolean onBeforeRequest(long sectionPos, CompletableFuture<ERequestResult> future) { return true; }
@Override
protected String getQueueName() { return "Sync On Login Queue"; }
@@ -18,7 +18,7 @@ public class SessionConfig implements INetworkObject
private static final LinkedHashMap<String, Entry> CONFIG_ENTRIES = new LinkedHashMap<>();
private final LinkedHashMap<String, Object> values = new LinkedHashMap<>();
private final HashMap<String, Object> values = new HashMap<>();
public SessionConfig constrainingConfig;
@@ -33,9 +33,9 @@ public class SessionConfig implements INetworkObject
registerConfigEntry(Config.Common.WorldGenerator.enableDistantGeneration, Boolean::logicalAnd);
registerConfigEntry(Config.Server.maxGenerationRequestDistance, Math::min);
registerConfigEntry(Config.Server.generationBoundsX, (x, y) -> x);
registerConfigEntry(Config.Server.generationBoundsZ, (x, y) -> x);
registerConfigEntry(Config.Server.generationBoundsRadius, (x, y) -> x);
registerConfigEntry(Config.Server.generationBoundsX, (x, y) -> y);
registerConfigEntry(Config.Server.generationBoundsZ, (x, y) -> y);
registerConfigEntry(Config.Server.generationBoundsRadius, (x, y) -> y);
registerConfigEntry(Config.Server.generationRequestRateLimit, Math::min);
registerConfigEntry(Config.Server.enableRealTimeUpdates, Boolean::logicalAnd);
@@ -119,10 +119,17 @@ public class SessionConfig implements INetworkObject
}
return (this.constrainingConfig != null
? (T) entry.valueConstrainer.apply(value, this.constrainingConfig.getValue(name))
? (T) entry.valueConstrainer.apply(this.constrainingConfig.getValue(name), value)
: value);
}
public <T> void constrainValue(ConfigEntry<T> configEntry, T value) { this.constrainValue(configEntry.getChatCommandName(), value); }
private void constrainValue(String name, Object value)
{
Entry entry = CONFIG_ENTRIES.get(name);
this.values.put(name, entry.valueConstrainer.apply(this.getValue(name), value));
}
private Map<String, ?> getValues()
{
return CONFIG_ENTRIES.keySet().stream().collect(Collectors.toMap(
@@ -34,8 +34,6 @@ public class FullDataPayloadReceiver implements AutoCloseable
})
.build().asMap();
@Override
public void close()
{
@@ -69,19 +67,21 @@ public class FullDataPayloadReceiver implements AutoCloseable
});
}
public FullDataSourceV2DTO decodeDataSourceAndReleaseBuffer(FullDataPayload msg)
public FullDataSourceV2DTO decodeDataSourceAndReleaseBuffer(FullDataPayload payload)
{
CompositeByteBuf compositeByteBuffer = this.buffersById.get(msg.dtoBufferId);
CompositeByteBuf compositeByteBuffer = this.buffersById.get(payload.dtoBufferId);
LodUtil.assertTrue(compositeByteBuffer != null);
try
{
return INetworkObject.decodeToInstance(FullDataSourceV2DTO.CreateEmptyDataSourceForDecoding(), compositeByteBuffer);
FullDataSourceV2DTO dataSourceDto = INetworkObject.decodeToInstance(FullDataSourceV2DTO.CreateEmptyDataSourceForDecoding(), compositeByteBuffer);
LOGGER.debug("Buffer {} DTO: {}", payload.dtoBufferId, dataSourceDto);
return dataSourceDto;
}
finally
{
// Releasing the buffer is handled by cache
this.buffersById.remove(msg.dtoBufferId);
this.buffersById.remove(payload.dtoBufferId);
}
}
@@ -13,7 +13,7 @@ import java.util.function.*;
public class FullDataPayloadSender implements AutoCloseable
{
private static final int TICK_RATE = 4;
private static final int TICK_RATE = 20;
/** 1 Mebibyte minus 576 bytes for other info */
public static final int FULL_DATA_SPLIT_SIZE_IN_BYTES = 1_048_000;
@@ -2,6 +2,7 @@ package com.seibel.distanthorizons.core.multiplayer.server;
import com.seibel.distanthorizons.api.enums.worldGeneration.EDhApiDistantGeneratorMode;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.file.fullDatafile.GeneratedFullDataSourceProvider;
import com.seibel.distanthorizons.core.level.AbstractDhServerLevel;
import com.seibel.distanthorizons.core.logging.ConfigBasedLogger;
@@ -59,41 +60,82 @@ public class FullDataSourceRequestHandler
}
// the client timestamp will be null if we want to retrieve the LOD regardless of when it was last updated
long clientTimestamp = (message.clientTimestamp != null) ? message.clientTimestamp : -1;
// the server timestamp will be null if no LOD data exists for this position
Long serverTimestamp = this.fullDataSourceProvider().getTimestampForPos(message.sectionPos);
if (serverTimestamp == null
|| serverTimestamp <= clientTimestamp)
AbstractExecutorService fileHandlerExecutor = ThreadPoolUtil.getFileHandlerExecutor();
if (fileHandlerExecutor == null)
{
// either no data exists to sync, or the client is already up to date
rateLimiterSet.syncOnLoginRateLimiter.release();
message.sendResponse(new FullDataSourceResponseMessage(null));
// shouldn't normally happen, but just in case
LOGGER.warn("Unable to send FullDataSourceResponseMessage - getFileHandlerExecutor() is null");
return;
}
AbstractExecutorService executor = ThreadPoolUtil.getNetworkCompressionExecutor();
if (executor == null)
AbstractExecutorService networkCompressionExecutor = ThreadPoolUtil.getNetworkCompressionExecutor();
if (networkCompressionExecutor == null)
{
// shouldn't normally happen, but just in case
LOGGER.warn("Unable to send FullDataSourceResponseMessage - getNetworkCompressionExecutor() is null");
return;
}
this.fullDataSourceProvider().getAsync(message.sectionPos).thenAcceptAsync(fullDataSource ->
{
try (FullDataPayload payload = new FullDataPayload(fullDataSource, this.getAllBeamsForPos(message.sectionPos)))
// get the data requested by the client
CompletableFuture<FullDataSourceV2> getServerDatasourceFuture = CompletableFuture.supplyAsync(() ->
{
fullDataSource.close();
serverPlayerState.fullDataPayloadSender.sendInChunks(payload, () ->
try
{
message.sendResponse(new FullDataSourceResponseMessage(payload));
rateLimiterSet.syncOnLoginRateLimiter.release();
});
}
}, executor);
// the client timestamp will be null if we want to retrieve the LOD regardless of when it was last updated
long clientTimestamp = (message.clientTimestamp != null) ? message.clientTimestamp : -1;
// the server timestamp will be null if no LOD data exists for this position
Long serverTimestamp = this.fullDataSourceProvider().getTimestampForPos(message.sectionPos);
if (serverTimestamp == null
|| serverTimestamp <= clientTimestamp)
{
// either no data exists to sync, or the client is already up to date
rateLimiterSet.syncOnLoginRateLimiter.release();
message.sendResponse(new FullDataSourceResponseMessage(null));
return null;
}
// get the server's datasource
return this.fullDataSourceProvider().get(message.sectionPos);
}
catch (Exception e)
{
LOGGER.error("Unexpected issue getting server-side LOD for request at pos [" + DhSectionPos.toString(message.sectionPos) + "], error: [" + e.getMessage() + "].", e);
return null;
}
}, fileHandlerExecutor);
// send the found data
getServerDatasourceFuture.thenAcceptAsync(fullDataSource ->
{
try
{
// no server data source found
if (fullDataSource == null)
{
return;
}
// send the found data source to client
try (FullDataPayload payload = new FullDataPayload(fullDataSource, this.getAllBeamsForPos(message.sectionPos)))
{
fullDataSource.close();
serverPlayerState.fullDataPayloadSender.sendInChunks(payload, () ->
{
message.sendResponse(new FullDataSourceResponseMessage(payload));
rateLimiterSet.syncOnLoginRateLimiter.release();
});
}
}
catch (Exception e)
{
LOGGER.error("Unexpected issue sending request for pos [" + DhSectionPos.toString(message.sectionPos) + "], error: [" + e.getMessage() + "].", e);
}
}, networkCompressionExecutor);
}
public void queueWorldGenForRequestMessage(ServerPlayerState serverPlayerState, FullDataSourceRequestMessage message, ServerPlayerState.RateLimiterSet rateLimiterSet)
@@ -24,7 +24,7 @@ public class ServerPlayerState implements Closeable
{
private final ConfigChangeListener<String> levelKeyPrefixChangeListener
= new ConfigChangeListener<>(Config.Server.levelKeyPrefix, this::onLevelKeyPrefixConfigChanged);
private final SessionConfig.AnyChangeListener configAnyChangeListener = new SessionConfig.AnyChangeListener(this::onSessionConfigChanged);
private final SessionConfig.AnyChangeListener configAnyChangeListener = new SessionConfig.AnyChangeListener(this::sendConfigMessage);
private String lastLevelKey = "";
@@ -56,8 +56,9 @@ public class ServerPlayerState implements Closeable
this.networkSession.registerHandler(SessionConfigMessage.class, (sessionConfigMessage) ->
{
this.sessionConfig.constrainingConfig = sessionConfigMessage.config;
this.sendLevelKey();
this.networkSession.sendMessage(new SessionConfigMessage(this.sessionConfig));
this.sendConfigMessage();
});
this.networkSession.registerHandler(CloseInternalEvent.class, event -> {
@@ -93,7 +94,14 @@ public class ServerPlayerState implements Closeable
}
}
private void onSessionConfigChanged() { this.networkSession.sendMessage(new SessionConfigMessage(this.sessionConfig)); }
private void sendConfigMessage()
{
double coordinateScale = this.getServerPlayer().getLevel().getDimensionType().getCoordinateScale();
this.sessionConfig.constrainValue(Config.Server.generationBoundsX, (int) (Config.Server.generationBoundsX.get() / coordinateScale));
this.sessionConfig.constrainValue(Config.Server.generationBoundsZ, (int) (Config.Server.generationBoundsZ.get() / coordinateScale));
this.networkSession.sendMessage(new SessionConfigMessage(this.sessionConfig));
}
@@ -46,6 +46,13 @@ public class ServerPlayerStateManager
public void handlePluginMessage(IServerPlayerWrapper player, AbstractNetworkMessage message)
{
// done to prevent a rare null-pointer on Neo/Forge
if (player == null || message == null)
{
return;
}
MessageQueueState messageQueue = this.messageQueueByPlayerWrapper.computeIfAbsent(player, k -> new MessageQueueState());
messageQueue.messageQueue.add(message);
@@ -5,10 +5,12 @@ import com.seibel.distanthorizons.coreapi.util.StringUtil;
import it.unimi.dsi.fastutil.bytes.ByteArrayList;
import it.unimi.dsi.fastutil.longs.LongArrayList;
import it.unimi.dsi.fastutil.shorts.ShortArrayList;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import java.lang.ref.SoftReference;
import java.util.ArrayList;
import java.util.concurrent.atomic.AtomicBoolean;
/**
* This keeps track of all the poolable
@@ -20,7 +22,16 @@ import java.util.ArrayList;
public class PhantomArrayListCheckout implements AutoCloseable
{
/** defines which pool the arrays should be returned too */
@NotNull
private final PhantomArrayListPool owningPool;
/**
* soft reference used by the {@link PhantomArrayListPool} so this checkout can be
* freed if there isn't enough memory.
*/
@NotNull
public final SoftReference<PhantomArrayListCheckout> ownerSoftReference;
/** Will be null if the parent pool doesn't want leak stack tracing */
@Nullable
public final String allocationStackTrace;
@@ -28,7 +39,6 @@ public class PhantomArrayListCheckout implements AutoCloseable
private final ArrayList<ByteArrayList> byteArrayLists = new ArrayList<>();
private final ArrayList<ShortArrayList> shortArrayLists = new ArrayList<>();
private final ArrayList<LongArrayList> longArrayLists = new ArrayList<>();
private final ArrayList<SoftReference<LongArrayList>> longArrayRefLists = new ArrayList<>();
@@ -36,7 +46,7 @@ public class PhantomArrayListCheckout implements AutoCloseable
// constructor //
//=============//
public PhantomArrayListCheckout(PhantomArrayListPool owningPool)
public PhantomArrayListCheckout(@NotNull PhantomArrayListPool owningPool)
{
if (owningPool.logGarbageCollectedStacks)
{
@@ -50,6 +60,7 @@ public class PhantomArrayListCheckout implements AutoCloseable
}
this.owningPool = owningPool;
this.ownerSoftReference = new SoftReference<>(this);
}
@@ -60,11 +71,7 @@ public class PhantomArrayListCheckout implements AutoCloseable
public void addByteArrayList(ByteArrayList list) { this.byteArrayLists.add(list); }
public void addShortArrayList(ShortArrayList list) { this.shortArrayLists.add(list); }
public void addLongArrayListRef(LongArrayList list, SoftReference<LongArrayList> listRef)
{
this.longArrayLists.add(list);
this.longArrayRefLists.add(listRef);
}
public void addLongArrayListRef(LongArrayList list) { this.longArrayLists.add(list); }
@@ -100,7 +107,6 @@ public class PhantomArrayListCheckout implements AutoCloseable
public ArrayList<ByteArrayList> getAllByteArrays() { return this.byteArrayLists; }
public ArrayList<ShortArrayList> getAllShortArrays() { return this.shortArrayLists; }
public ArrayList<LongArrayList> getAllLongArrays() { return this.longArrayLists; }
public ArrayList<SoftReference<LongArrayList>> getAllLongArrayRefs() { return this.longArrayRefLists; }
@@ -109,10 +115,7 @@ public class PhantomArrayListCheckout implements AutoCloseable
//================//
@Override
public void close()
{
this.owningPool.returnCheckout(this);
}
public void close() { this.owningPool.returnCheckout(this); }
@@ -28,7 +28,7 @@ public abstract class PhantomArrayListParent implements AutoCloseable
* It's recommended to set this as null after the child's constructor
* finishes to show the pooled arrays have all been accessed
*/
protected PhantomArrayListCheckout pooledArraysCheckout;
protected final PhantomArrayListCheckout pooledArraysCheckout;
@@ -57,19 +57,7 @@ public abstract class PhantomArrayListParent implements AutoCloseable
//================//
@Override
public void close() //throws Exception
{
try
{
this.phantomReference.clear();
PhantomArrayListCheckout checkout = this.phantomArrayListPool.phantomRefToCheckout.remove(this.phantomReference);
this.phantomArrayListPool.returnCheckout(checkout);
}
catch (Exception e)
{
LOGGER.error("Unable to close Phantom Array", e);
}
}
public void close() { this.phantomArrayListPool.returnParentPhantomRef(this.phantomReference); }
}
@@ -4,7 +4,6 @@ import com.seibel.distanthorizons.core.api.internal.ClientApi;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.logging.f3.F3Screen;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.util.ThreadUtil;
import com.seibel.distanthorizons.core.util.objects.Pair;
import com.seibel.distanthorizons.coreapi.ModInfo;
@@ -14,7 +13,9 @@ import it.unimi.dsi.fastutil.longs.LongArrayList;
import it.unimi.dsi.fastutil.shorts.ShortArrayList;
import org.apache.logging.log4j.Logger;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import java.lang.ref.PhantomReference;
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.SoftReference;
@@ -25,8 +26,6 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiConsumer;
import java.util.function.Supplier;
/**
* DH uses a lot of potentially large arrays of {@link Byte}s and {@link Long}s.
@@ -46,6 +45,12 @@ import java.util.function.Supplier;
* This is less efficient since it may allow a lot of additional arrays to
* be created while we wait for the garbage collector to run, but
* does prevent any leaks from {@link PhantomArrayListParent} that weren't closed.
*
* <br><br>
* <strong>Use Notes: </strong><br>
* If possible all checkouts for a given pool should be the same size,
* since {@link PhantomArrayListCheckout}'s are shared, getting the same size checkout each time
* prevents accidentally returning a larger checkout than necessary, which wastes memory.
*/
public class PhantomArrayListPool
{
@@ -83,10 +88,7 @@ public class PhantomArrayListPool
public final ReferenceQueue<PhantomArrayListParent> phantomRefQueue = new ReferenceQueue<>();
private final ConcurrentLinkedQueue<ByteArrayList> pooledByteArrays = new ConcurrentLinkedQueue<>();
private final ConcurrentLinkedQueue<ShortArrayList> pooledShortArrays = new ConcurrentLinkedQueue<>();
private final ConcurrentLinkedQueue<SoftReference<LongArrayList>> pooledLongArrays = new ConcurrentLinkedQueue<>();
private final ConcurrentLinkedQueue<SoftReference<PhantomArrayListCheckout>> pooledCheckoutsRefs = new ConcurrentLinkedQueue<>();
/** counts how many byte arrays have been created by this pool */
private final AtomicInteger totalByteArrayCountRef = new AtomicInteger(0);
@@ -102,6 +104,15 @@ public class PhantomArrayListPool
/** used for debugging, represents an estimate for how many bytes the long[] pool contains */
private long lastLongPoolSizeInBytes = -1;
/** used for debugging, represents an estimate for how many byte[]'s are currently in this pool*/
private int lastBytePoolCount = 0;
/** used for debugging, represents an estimate for how many short[]'s are currently in this pool*/
private int lastShortPoolCount = 0;
/** used for debugging, represents an estimate for how many long[]'s are currently in this pool*/
private int lastLongPoolCount = 0;
/** used for debugging, represents an estimate for how many checkouts are currently in this pool*/
private int lastCheckoutPoolCount = 0;
/** For pools backed by {@link SoftReference}'s we may need to decrease the size when elements are garbage collected */
private boolean clearLastRefPoolSizes = false;
@@ -132,30 +143,77 @@ public class PhantomArrayListPool
// get checkout //
//==============//
/**
* If possible all checkouts for a given pool should be the same size,
* since {@link PhantomArrayListCheckout}'s are shared, returning the same size
* prevents accidentally returning a larger checkout than necessary, which wastes memory.
*/
public PhantomArrayListCheckout checkoutArrays(int byteArrayCount, int shortArrayCount, int longArrayCount)
{
PhantomArrayListCheckout checkout = new PhantomArrayListCheckout(this);
PhantomArrayListCheckout checkout = null;
while (checkout == null)
{
SoftReference<PhantomArrayListCheckout> checkoutRef = this.pooledCheckoutsRefs.poll();
if (checkoutRef == null)
{
// pool is empty, create new checkout
checkout = new PhantomArrayListCheckout(this);
}
else
{
checkout = checkoutRef.get();
if (checkout != null)
{
// use pooled checkout
}
else
{
// this reference is pointing to null,
// the checkout must have been garbage collected,
// that means we don't have enough memory
if (!lowMemoryWarningLogged)
{
lowMemoryWarningLogged = true;
// orange text
String message = "\u00A76" + "Distant Horizons: Insufficient memory detected." + "\u00A7r \n" +
"This may cause stuttering or crashing. \n" +
"Potential causes: \n" +
"1. your allocated memory isn't high enough \n" +
"2. your DH CPU preset is too high \n" +
"3. your DH quality preset is too high";
LOGGER.warn(message);
if (Config.Common.Logging.Warning.showPoolInsufficientMemoryWarning.get())
{
ClientApi.INSTANCE.showChatMessageNextFrame(message);
}
}
this.clearLastRefPoolSizes = true;
}
}
}
// get any missing arrays
// byte
for (int i = 0; i < byteArrayCount; i++)
for (int i = checkout.getByteArrayCount(); i < byteArrayCount; i++)
{
checkout.addByteArrayList(getPooledArray(this.pooledByteArrays, () -> this.createEmptyByteArrayList()));
checkout.addByteArrayList(this.createEmptyByteArrayList());
}
// short
for (int i = 0; i < shortArrayCount; i++)
for (int i = checkout.getShortArrayCount(); i < shortArrayCount; i++)
{
checkout.addShortArrayList(getPooledArray(this.pooledShortArrays, () -> this.createEmptyShortArrayList()));
checkout.addShortArrayList(this.createEmptyShortArrayList());
}
// long
for (int i = 0; i < longArrayCount; i++)
for (int i = checkout.getLongArrayCount(); i < longArrayCount; i++)
{
addRefPooledArray(
this.pooledLongArrays,
this::createEmptyLongArrayList,
this::onLongArrayListGarbageCollected,
checkout::addLongArrayListRef);
checkout.addLongArrayListRef(this.createEmptyLongArrayList());
}
return checkout;
@@ -184,91 +242,6 @@ public class PhantomArrayListPool
}
// garbage collection handlers //
/** should only happen if Java doesn't have enough memory */
private void onLongArrayListGarbageCollected()
{
this.clearLastRefPoolSizes = true;
this.totalLongArrayCountRef.getAndDecrement();
}
// internal pool handlers //
private static <T extends List<?>> T getPooledArray(ConcurrentLinkedQueue<T> pool, Supplier<T> emptyArrayCreatorFunc)
{
T array = pool.poll();
if (array != null)
{
array.clear();
return array;
}
else
{
// no pooled sources exist
return emptyArrayCreatorFunc.get();
}
}
private static <T extends List<?>> void addRefPooledArray(
ConcurrentLinkedQueue<SoftReference<T>> arrayPool,
Supplier<T> emptyArrayCreatorFunc,
Runnable arrayGarbageCollectedFunc,
BiConsumer<T, SoftReference<T>> putArrayFunc)
{
T array = null;
SoftReference<T> arrayRef = arrayPool.poll();
// find the first non-null pooled array
while (arrayRef != null && array == null)
{
array = arrayRef.get();
if (array == null)
{
// this reference is pointing to null,
// the array must have been garbage collected,
// that means we don't have enough memory
if (!lowMemoryWarningLogged)
{
lowMemoryWarningLogged = true;
// orange text
String message = "\u00A76" + "Distant Horizons: Insufficient memory detected." + "\u00A7r \n" +
"This may cause stuttering or crashing. \n" +
"Either: your allocated memory isn't high enough, \n" +
"your DH CPU preset is too high, or your DH quality preset is too high.";
LOGGER.warn(message);
if (Config.Common.Logging.Warning.showPoolInsufficientMemoryWarning.get())
{
ClientApi.INSTANCE.showChatMessageNextFrame(message);
}
}
arrayGarbageCollectedFunc.run();
// try the next reference
arrayRef = arrayPool.poll();
}
}
if (array != null)
{
LodUtil.assertTrue(arrayRef != null, "How did we get an array without it's reference?");
array.clear();
}
else
{
// no pooled sources exist
array = emptyArrayCreatorFunc.get();
arrayRef = new SoftReference<>(array);
}
putArrayFunc.accept(array, arrayRef);
}
//==================//
// phantom recovery //
@@ -333,9 +306,9 @@ public class PhantomArrayListPool
{
// we only want to log when something has been returned
if (checkoutCount != 0
|| returnedByteArrayCount != 0
|| returnedShortArrayCount != 0
|| returnedLongArrayCount != 0)
|| returnedByteArrayCount != 0
|| returnedShortArrayCount != 0
|| returnedLongArrayCount != 0)
{
LOGGER.warn("Pool: ["+ pool.name+"] phantom recovery. Returned checkouts:["+F3Screen.NUMBER_FORMAT.format(checkoutCount)+"], byte:["+F3Screen.NUMBER_FORMAT.format(returnedByteArrayCount)+"], short:["+F3Screen.NUMBER_FORMAT.format(returnedShortArrayCount)+"], long:["+F3Screen.NUMBER_FORMAT.format(returnedLongArrayCount)+"].");
@@ -405,21 +378,29 @@ public class PhantomArrayListPool
// return checkout //
//=================//
public void returnCheckout(PhantomArrayListCheckout checkout)
public void returnParentPhantomRef(@NotNull PhantomReference<PhantomArrayListParent> parentRef)
{
try
{
parentRef.clear();
// will be null if the this parent has already been returned
PhantomArrayListCheckout checkout = this.phantomRefToCheckout.remove(parentRef);
this.returnCheckout(checkout);
}
catch (Exception e)
{
LOGGER.error("Unable to close Phantom Array, error: ["+e.getMessage()+"].", e);
}
}
public void returnCheckout(@Nullable PhantomArrayListCheckout checkout)
{
if (checkout == null)
{
throw new IllegalArgumentException("Null phantom checkout, object is being closed multiple times.");
}
// In James' testing pooling the checkout object wasn't necessary
// since it is relatively small and short lived, thus
// the GC can handle quickly discarding it.
this.pooledByteArrays.addAll(checkout.getAllByteArrays());
this.pooledShortArrays.addAll(checkout.getAllShortArrays());
this.pooledLongArrays.addAll(checkout.getAllLongArrayRefs());
SoftReference<PhantomArrayListCheckout> checkoutRef = checkout.ownerSoftReference;
this.pooledCheckoutsRefs.add(checkoutRef);
//LOGGER.info("Returned ["+checkout.byteArrayLists.size()+"/"+this.pooledByteArrays.size()+"] bytes and ["+checkout.longArrayLists.size()+"/"+this.pooledLongArrays.size()+"] longs.");\
}
@@ -444,9 +425,9 @@ public class PhantomArrayListPool
totalShortArrayCount += pool.totalShortArrayCountRef.get();
totalLongArrayCount += pool.totalLongArrayCountRef.get();
pooledByteArraySize += pool.pooledByteArrays.size();
pooledShortArraySize += pool.pooledShortArrays.size();
pooledLongArraySize += pool.pooledLongArrays.size();
pooledByteArraySize += pool.lastBytePoolCount;
pooledShortArraySize += pool.lastShortPoolCount;
pooledLongArraySize += pool.lastLongPoolCount;
lastBytePoolSizeInBytes += pool.lastBytePoolSizeInBytes;
lastShortPoolSizeInBytes += pool.lastShortPoolSizeInBytes;
@@ -475,7 +456,7 @@ public class PhantomArrayListPool
addDebugMenuStringsToList(messageList,
this.name,
this.totalByteArrayCountRef.get(), this.totalShortArrayCountRef.get(), this.totalLongArrayCountRef.get(),
this.pooledByteArrays.size(), this.pooledShortArrays.size(), this.pooledLongArrays.size(),
this.lastBytePoolCount, this.lastShortPoolCount, this.lastLongPoolCount,
this.lastBytePoolSizeInBytes, this.lastShortPoolSizeInBytes, this.lastLongPoolSizeInBytes
);
}
@@ -529,25 +510,60 @@ public class PhantomArrayListPool
*/
public void recalculateSizeForDebugging()
{
// byte
long bytePoolByteSize = estimateMemoryUsage(this.pooledByteArrays, Byte.BYTES);
this.lastBytePoolSizeInBytes = Math.max(bytePoolByteSize, this.lastBytePoolSizeInBytes);
long bytePoolByteSize = 0;
long shortPoolByteSize = 0;
long longPoolByteSize = 0;
// short
long shortPoolByteSize = estimateMemoryUsage(this.pooledShortArrays, Short.BYTES);
this.lastShortPoolSizeInBytes = Math.max(shortPoolByteSize, this.lastShortPoolSizeInBytes);
int bytePoolCount = 0;
int shortPoolCount = 0;
int longPoolCount = 0;
// long
// checkouts //
for (SoftReference<PhantomArrayListCheckout> pooledCheckoutRef : this.pooledCheckoutsRefs)
{
PhantomArrayListCheckout pooledCheckout = pooledCheckoutRef.get();
if (pooledCheckout == null)
{
continue;
}
bytePoolByteSize += estimateMemoryUsage(pooledCheckout.getAllByteArrays(), Byte.BYTES);
bytePoolCount += pooledCheckout.getAllByteArrays().size();
shortPoolByteSize += estimateMemoryUsage(pooledCheckout.getAllShortArrays(), Short.BYTES);
shortPoolCount += pooledCheckout.getAllShortArrays().size();
longPoolByteSize += estimateMemoryUsage(pooledCheckout.getAllLongArrays(), Long.BYTES);
longPoolCount += pooledCheckout.getAllLongArrays().size();
}
// clear old values if something was garbage collected
if (this.clearLastRefPoolSizes)
{
this.lastBytePoolSizeInBytes = 0;
this.lastShortPoolSizeInBytes = 0;
this.lastLongPoolSizeInBytes = 0;
this.clearLastRefPoolSizes = false;
}
long longPoolByteSize = estimateRefMemoryUsage(this.pooledLongArrays, Long.BYTES);
this.lastCheckoutPoolCount = this.pooledCheckoutsRefs.size();
// byte //
// math.max is used since the pool should only grow until a soft reference is freed,
// and it's easier to understand if this constantly grows instead of jumping around
this.lastBytePoolSizeInBytes = Math.max(bytePoolByteSize, this.lastBytePoolSizeInBytes);
this.lastBytePoolCount = bytePoolCount;
// short //
this.lastShortPoolSizeInBytes = Math.max(shortPoolByteSize, this.lastShortPoolSizeInBytes);
this.lastShortPoolCount = shortPoolCount;
// long //
this.lastLongPoolSizeInBytes = Math.max(longPoolByteSize, this.lastLongPoolSizeInBytes);
this.lastLongPoolCount = longPoolCount;
}
private static <T extends Collection<?>> long estimateMemoryUsage(ConcurrentLinkedQueue<T> pool, long elementSizeInBytes)
private static <T extends Collection<?>> long estimateMemoryUsage(Iterable<T> pool, long elementSizeInBytes)
{
long longByteSize = 0;
for (T array : pool)
@@ -83,6 +83,19 @@ public class DhChunkPos
public int getMinBlockX() { return this.x << 4; }
public int getMinBlockZ() { return this.z << 4; }
public int getMaxBlockX()
{
int minBlockPos = this.getMinBlockX() + LodUtil.CHUNK_WIDTH;
minBlockPos += (minBlockPos < 0) ? -1 : 0;
return minBlockPos;
}
public int getMaxBlockZ()
{
int minBlockPos = this.getMinBlockZ() + LodUtil.CHUNK_WIDTH;
minBlockPos += (minBlockPos < 0) ? -1 : 0;
return minBlockPos;
}
public DhBlockPos2D getMinBlockPos() { return new DhBlockPos2D(this.x << 4, this.z << 4); }
public boolean contains(DhBlockPos pos)
@@ -92,8 +105,8 @@ public class DhChunkPos
int maxBlockX = minBlockX + LodUtil.CHUNK_WIDTH;
int maxBlockZ = minBlockZ + LodUtil.CHUNK_WIDTH;
return minBlockX <= pos.getX() && pos.getX() < maxBlockX
&& minBlockZ <= pos.getZ() && pos.getZ() < maxBlockZ;
return minBlockX >= pos.getX() && pos.getX() < maxBlockX
&& minBlockZ >= pos.getZ() && pos.getZ() < maxBlockZ;
}
public double distance(DhChunkPos other)
@@ -24,6 +24,7 @@ import com.google.common.cache.CacheBuilder;
import com.google.common.cache.RemovalCause;
import com.google.common.cache.RemovalNotification;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dataObjects.render.CachedColumnRenderSource;
import com.seibel.distanthorizons.core.dataObjects.render.ColumnRenderSource;
import com.seibel.distanthorizons.core.dataObjects.render.bufferBuilding.ColumnRenderBuffer;
import com.seibel.distanthorizons.core.enums.EDhDirection;
@@ -34,6 +35,8 @@ import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos2D;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.render.renderer.DebugRenderer;
import com.seibel.distanthorizons.core.render.renderer.IDebugRenderable;
import com.seibel.distanthorizons.core.render.renderer.generic.BeaconRenderHandler;
import com.seibel.distanthorizons.core.render.renderer.generic.GenericObjectRenderer;
import com.seibel.distanthorizons.core.util.KeyedLockContainer;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.util.ThreadUtil;
@@ -43,6 +46,7 @@ import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
import com.seibel.distanthorizons.coreapi.util.MathUtil;
import it.unimi.dsi.fastutil.longs.LongIterator;
import org.apache.logging.log4j.Logger;
import org.jetbrains.annotations.Nullable;
import javax.annotation.WillNotClose;
import java.awt.*;
@@ -51,6 +55,7 @@ import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantLock;
/**
@@ -95,48 +100,29 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
* caching the loaded positions significantly improves initial loading performance
* since the same position doesn't need to be loaded 5 times.
*/
private final Cache<Long, ColumnRenderSource> cachedRenderSourceByPos
private final Cache<Long, CachedColumnRenderSource> cachedRenderSourceByPos
= CacheBuilder.newBuilder()
// availableProcessors() : each process may need to be loading a render source
// +1 : add 1 thread count buffer to reduce the chance of accidentally unloading a render source before it's used
// *5 : each render source needs it's 4 adjacent sides, so a total of 5 render sources are needed per load
.maximumSize((Runtime.getRuntime().availableProcessors() + 1) * 5L)
.removalListener((RemovalNotification<Long, ColumnRenderSource> removalNotification) ->
{
RemovalCause cause = removalNotification.getCause();
if (cause == RemovalCause.EXPLICIT
|| cause == RemovalCause.EXPIRED
|| cause == RemovalCause.COLLECTED
|| cause == RemovalCause.SIZE)
{
// cleanup needs to be handled on a different thread to prevent locking up the main loading threads
ThreadPoolExecutor executor = ThreadPoolUtil.getCleanupExecutor();
executor.execute(() ->
{
// close the render source after it's been
ColumnRenderSource renderSource = removalNotification.getValue();
if (renderSource != null)
{
ReentrantLock lock = renderLoadLockContainer.getLockForPos(renderSource.getPos());
try
{
lock.lock();
renderSource.close();
}
finally
{
lock.unlock();
}
}
else
{
// shouldn't happen, but just in case
LOGGER.error("Unable to close null cached render source.");
}
});
}
})
.<Long, ColumnRenderSource>build();
// No closing logic since the CachedColumnRenderSource is in charge
// of freeing the underlying ColumnRenderSource.
// That way we don't have to worry about accidentally closing an in-use object.
.<Long, CachedColumnRenderSource>build();
/**
* Used to limit how many upload tasks are queued at once.
* If all the upload tasks are queued at once, they will start uploading nearest
* to the player, however if the player moves, that order is no longer valid and holes may appear
* as further sections are loaded before closer ones.
* Only queuing a few of the sections at a time solves this problem.
*/
public final AtomicInteger uploadTaskCountRef = new AtomicInteger(0);
@Nullable
public final BeaconRenderHandler beaconRenderHandler;
/** the smallest numerical detail level number that can be rendered */
@@ -167,6 +153,10 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
this.level = level;
this.fullDataSourceProvider = fullDataSourceProvider;
this.blockRenderDistanceDiameter = viewDiameterInBlocks;
GenericObjectRenderer genericObjectRenderer = this.level.getGenericRenderer();
this.beaconRenderHandler = (genericObjectRenderer != null) ? new BeaconRenderHandler(genericObjectRenderer) : null;
}
@@ -247,7 +237,7 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
long rootPos = rootPosIterator.nextLong();
if (this.getNode(rootPos) == null)
{
this.setValue(rootPos, new LodRenderSection(rootPos, this, this.level, this.fullDataSourceProvider, this.cachedRenderSourceByPos, this.renderLoadLockContainer));
this.setValue(rootPos, new LodRenderSection(rootPos, this, this.level, this.fullDataSourceProvider, this.uploadTaskCountRef, this.cachedRenderSourceByPos, this.renderLoadLockContainer));
}
QuadNode<LodRenderSection> rootNode = this.getNode(rootPos);
@@ -286,7 +276,7 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
// create the node
if (quadNode == null && this.isSectionPosInBounds(sectionPos)) // the position bounds should only fail when at the edge of the user's render distance
{
rootNode.setValue(sectionPos, new LodRenderSection(sectionPos, this, this.level, this.fullDataSourceProvider, this.cachedRenderSourceByPos, this.renderLoadLockContainer));
rootNode.setValue(sectionPos, new LodRenderSection(sectionPos, this, this.level, this.fullDataSourceProvider, this.uploadTaskCountRef, this.cachedRenderSourceByPos, this.renderLoadLockContainer));
quadNode = rootNode.getNode(sectionPos);
}
if (quadNode == null)
@@ -299,7 +289,7 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
LodRenderSection renderSection = quadNode.value;
if (renderSection == null)
{
renderSection = new LodRenderSection(sectionPos, this, this.level, this.fullDataSourceProvider, this.cachedRenderSourceByPos, this.renderLoadLockContainer);
renderSection = new LodRenderSection(sectionPos, this, this.level, this.fullDataSourceProvider, this.uploadTaskCountRef, this.cachedRenderSourceByPos, this.renderLoadLockContainer);
quadNode.setValue(sectionPos, renderSection);
}
@@ -340,9 +330,11 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
}
else
{
// onRenderingDisabled() needs to be fired before the children are enabled so beacons render correctly
// children are all loaded, unload this and parents
if (renderSection.getRenderingEnabled())
{
// needs to be fired before the children are enabled so beacons render correctly
renderSection.onRenderingDisabled();
@@ -407,14 +399,11 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
// prepare this section for rendering
if (!renderSection.gpuUploadInProgress()
&& renderSection.renderBuffer == null
&&
(
// this check is specifically for N-sized world generators where the higher quality
// data source may not exist yet, this is done to prevent holes while waiting for said generator
renderSection.getFullDataSourceExists()
// if we can't request generation we don't want to check for full data existing
// since that will prevent server LODs from loading high-detail LODs where quadrants haven't been generated.
|| !this.fullDataSourceProvider.canQueueRetrieval())
// TODO this is commented out since some users reported LODs refusing to
// load at their expected higher-detail levels
// this check is specifically for N-sized world generators where the higher quality
// data source may not exist yet, this is done to prevent holes while waiting for said generator
//&& renderSection.getFullDataSourceExists()
)
{
nodesNeedingLoading.add(renderSection);
@@ -463,7 +452,7 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
}
});
// onRenderingEnabled() needs to be fired after the children are disabled so beacons render correctly
// needs to be fired after the children are disabled so beacons render correctly
renderSection.onRenderingEnabled();
}
@@ -643,7 +632,22 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
*/
public void reloadPos(long pos)
{
this.cachedRenderSourceByPos.invalidate(pos); // TODO will this cause issues? we may need to lock this invalidation if the cached data source is currently in use
// clear cache //
this.clearRenderCacheForPos(pos);
for (EDhDirection direction : EDhDirection.ADJ_DIRECTIONS)
{
long adjacentPos = DhSectionPos.getAdjacentPos(pos, direction);
this.clearRenderCacheForPos(adjacentPos);
}
// queue reloads //
// only queue each section for reloading
// after the cache has been cleared,
// this is done to prevent accidentally using old cached data
this.sectionsToReload.add(pos);
// the adjacent locations also need to be updated to make sure lighting
@@ -652,10 +656,24 @@ public class LodQuadTree extends QuadTree<LodRenderSection> implements IDebugRen
for (EDhDirection direction : EDhDirection.ADJ_DIRECTIONS)
{
long adjacentPos = DhSectionPos.getAdjacentPos(pos, direction);
this.cachedRenderSourceByPos.invalidate(adjacentPos); // TODO will this cause issues? we may need to lock this invalidation if the cached data source is currently in use
this.sectionsToReload.add(adjacentPos);
}
}
private void clearRenderCacheForPos(long pos)
{
// locking is needed to prevent another thread
// from accessing the cache while it's being cleared
ReentrantLock lock = this.renderLoadLockContainer.getLockForPos(pos);
try
{
lock.lock();
this.cachedRenderSourceByPos.invalidate(pos);
}
finally
{
lock.unlock();
}
}
@@ -23,6 +23,7 @@ import com.google.common.base.Suppliers;
import com.google.common.cache.Cache;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dataObjects.fullData.sources.FullDataSourceV2;
import com.seibel.distanthorizons.core.dataObjects.render.CachedColumnRenderSource;
import com.seibel.distanthorizons.core.dataObjects.render.ColumnRenderSource;
import com.seibel.distanthorizons.core.dataObjects.render.bufferBuilding.ColumnRenderBufferBuilder;
import com.seibel.distanthorizons.core.dataObjects.render.bufferBuilding.LodQuadBuilder;
@@ -38,6 +39,9 @@ import com.seibel.distanthorizons.core.render.glObject.GLProxy;
import com.seibel.distanthorizons.core.render.renderer.IDebugRenderable;
import com.seibel.distanthorizons.core.dataObjects.render.bufferBuilding.ColumnRenderBuffer;
import com.seibel.distanthorizons.core.render.renderer.DebugRenderer;
import com.seibel.distanthorizons.core.render.renderer.generic.BeaconRenderHandler;
import com.seibel.distanthorizons.core.sql.dto.BeaconBeamDTO;
import com.seibel.distanthorizons.core.sql.repo.BeaconBeamRepo;
import com.seibel.distanthorizons.core.util.KeyedLockContainer;
import com.seibel.distanthorizons.core.util.threading.PriorityTaskPicker;
import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
@@ -49,6 +53,7 @@ import org.jetbrains.annotations.Nullable;
import javax.annotation.WillNotClose;
import java.awt.*;
import java.util.*;
import java.util.List;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantLock;
@@ -63,15 +68,6 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
private static final Logger LOGGER = DhLoggerBuilder.getLogger();
private static final IMinecraftClientWrapper MC = SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class);
/**
* Used to limit how many upload tasks are queued at once.
* If all the upload tasks are queued at once, they will start uploading nearest
* to the player, however if the player moves, that order is no longer valid and holes may appear
* as further sections are loaded before closer ones.
* Only queuing a few of the sections at a time solves this problem.
*/
public static final AtomicInteger GLOBAL_UPLOAD_TASKS_COUNT_REF = new AtomicInteger(0);
public final long pos;
@@ -81,8 +77,18 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
private final FullDataSourceProviderV2 fullDataSourceProvider;
private final LodQuadTree quadTree;
private final KeyedLockContainer<Long> renderLoadLockContainer;
private final Cache<Long, ColumnRenderSource> cachedRenderSourceByPos;
private final Cache<Long, CachedColumnRenderSource> cachedRenderSourceByPos;
private final AtomicInteger uploadTaskCountRef;
/**
* contains the list of beacons currently being rendered in this section
* if this list is modified the {@link LodRenderSection#beaconRenderHandler} should be updated to match.
*/
private final List<BeaconBeamDTO> activeBeaconList = new ArrayList<>();
@Nullable
public final BeaconRenderHandler beaconRenderHandler;
@Nullable
public final BeaconBeamRepo beaconBeamRepo;
private boolean renderingEnabled = false;
@@ -112,10 +118,15 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
*/
private CompletableFuture<ColumnRenderBuffer> bufferUploadFuture = null;
/** should be an empty array if no positions need to be generated */
/**
* should be an empty array if no positions need to be generated
*
* @deprecated see the comment where this variable is set
*/
@Nullable
private Supplier<LongArrayList> missingGenerationPos;
private LongArrayList getMissingGenerationPos() { return this.missingGenerationPos != null ? this.missingGenerationPos.get() : null; }
@Deprecated
private Supplier<LongArrayList> missingGenerationPosFunc;
private LongArrayList getMissingGenerationPos() { return this.missingGenerationPosFunc != null ? this.missingGenerationPosFunc.get() : null; }
private boolean checkedIfFullDataSourceExists = false;
private boolean fullDataSourceExists = false;
@@ -129,8 +140,9 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
public LodRenderSection(
long pos,
LodQuadTree quadTree,
IDhClientLevel level, FullDataSourceProviderV2 fullDataSourceProvider,
Cache<Long, ColumnRenderSource> cachedRenderSourceByPos, KeyedLockContainer<Long> renderLoadLockContainer)
IDhClientLevel level, FullDataSourceProviderV2 fullDataSourceProvider,
AtomicInteger uploadTaskCountRef,
Cache<Long, CachedColumnRenderSource> cachedRenderSourceByPos, KeyedLockContainer<Long> renderLoadLockContainer)
{
this.pos = pos;
this.quadTree = quadTree;
@@ -138,6 +150,10 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
this.renderLoadLockContainer = renderLoadLockContainer;
this.level = level;
this.fullDataSourceProvider = fullDataSourceProvider;
this.uploadTaskCountRef = uploadTaskCountRef;
this.beaconRenderHandler = this.quadTree.beaconRenderHandler;
this.beaconBeamRepo = this.level.getBeaconBeamRepo();
DebugRenderer.register(this, Config.Client.Advanced.Debugging.DebugWireframe.showRenderSectionStatus);
}
@@ -174,32 +190,35 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
// this means the closer (higher priority) tasks will load first.
// This also prevents issues where the nearby tasks are canceled due to
// LOD detail level changing, and having holes in the world
if (GLOBAL_UPLOAD_TASKS_COUNT_REF.getAndIncrement() > executor.getPoolSize())
if (this.uploadTaskCountRef.getAndIncrement() > executor.getPoolSize())
{
GLOBAL_UPLOAD_TASKS_COUNT_REF.decrementAndGet();
this.uploadTaskCountRef.decrementAndGet();
return false;
}
try
{
CompletableFuture<Void> future = new CompletableFuture<>();
this.getAndBuildRenderDataFuture = future;
this.getAndBuildRenderDataFuture = future; // TODO should use a setter/getter to guard against replacing an incomplete future
future.handle((voidObj, throwable) ->
{
// this has to fire are the end of every added future, otherwise we'll lock up and nothing will load
GLOBAL_UPLOAD_TASKS_COUNT_REF.decrementAndGet();
this.uploadTaskCountRef.decrementAndGet(); // TODO there is an issue where this variable isn't decremented properly, preventing LODs from loading in, or loading much slower
return null;
});
this.getAndBuildRenderDataRunnable = () ->
{
this.getAndUploadRenderDataToGpu();
// the future is passed in separate to prevent any possible race condition null pointers
future.complete(null);
// the task is done, we don't need to track these anymore
this.getAndBuildRenderDataFuture = null;
this.getAndBuildRenderDataRunnable = null;
this.getAndRefreshRenderingBeacons();
this.getAndUploadRenderDataToGpuAsync()
.thenRun(() ->
{
// the future is passed in separately (IE not using the local var) to prevent any possible race condition null pointers
future.complete(null);
// the task is done, we don't need to track these anymore
this.getAndBuildRenderDataFuture = null;
this.getAndBuildRenderDataRunnable = null;
});
};
executor.execute(this.getAndBuildRenderDataRunnable);
@@ -215,83 +234,138 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
return false;
}
}
private void getAndUploadRenderDataToGpu()
private CompletableFuture<Void> getAndUploadRenderDataToGpuAsync()
{
try
{
ColumnRenderSource renderSource = this.getRenderSourceForPos(this.pos);
if (renderSource == null)
// get the center pos data
return this.getRenderSourceForPosAsync(this.pos)
.thenCompose((CachedColumnRenderSource cachedRenderSource) ->
{
// nothing needs to be rendered
// TODO how doesn't this cause infinite file handler loops?
// to trigger an upload we check if the buffer is null, and we aren't
// setting the render buffer here
return;
}
boolean enableTransparency = Config.Client.Advanced.Graphics.Quality.transparency.get().transparencyEnabled;
LodQuadBuilder lodQuadBuilder = new LodQuadBuilder(enableTransparency, this.level.getClientLevelWrapper());
// load adjacent render sources
{
ColumnRenderSource northRenderSource = this.getRenderSourceForPos(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.NORTH));
ColumnRenderSource southRenderSource = this.getRenderSourceForPos(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.SOUTH));
ColumnRenderSource eastRenderSource = this.getRenderSourceForPos(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.EAST));
ColumnRenderSource westRenderSource = this.getRenderSourceForPos(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.WEST));
ColumnRenderSource[] adjacentRenderSections = new ColumnRenderSource[EDhDirection.ADJ_DIRECTIONS.length];
adjacentRenderSections[EDhDirection.NORTH.ordinal() - 2] = northRenderSource;
adjacentRenderSections[EDhDirection.SOUTH.ordinal() - 2] = southRenderSource;
adjacentRenderSections[EDhDirection.EAST.ordinal() - 2] = eastRenderSource;
adjacentRenderSections[EDhDirection.WEST.ordinal() - 2] = westRenderSource;
boolean[] adjIsSameDetailLevel = new boolean[EDhDirection.ADJ_DIRECTIONS.length];
adjIsSameDetailLevel[EDhDirection.NORTH.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.NORTH);
adjIsSameDetailLevel[EDhDirection.SOUTH.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.SOUTH);
adjIsSameDetailLevel[EDhDirection.EAST.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.EAST);
adjIsSameDetailLevel[EDhDirection.WEST.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.WEST);
// the render sources are only needed in this synchronous method,
// then they can be closed
ColumnRenderBufferBuilder.makeLodRenderData(lodQuadBuilder, renderSource, this.level, adjacentRenderSections, adjIsSameDetailLevel);
}
this.uploadToGpuAsync(lodQuadBuilder);
}
catch (Exception e)
{
LOGGER.error("Unexpected error while loading LodRenderSection ["+DhSectionPos.toString(this.pos)+"], Error: [" + e.getMessage() + "].", e);
}
try
{
if (cachedRenderSource == null || cachedRenderSource.columnRenderSource == null)
{
// nothing needs to be rendered
// TODO how doesn't this cause infinite file handler loops?
// to trigger an upload we check if the buffer is null, and we aren't
// setting the render buffer here
return CompletableFuture.completedFuture(null);
}
ColumnRenderSource thisRenderSource = cachedRenderSource.columnRenderSource;
boolean enableTransparency = Config.Client.Advanced.Graphics.Quality.transparency.get().transparencyEnabled;
LodQuadBuilder lodQuadBuilder = new LodQuadBuilder(enableTransparency, this.level.getClientLevelWrapper());
// get the adjacent positions
// needs to be done async to prevent threads waiting on the same positions to be processed
final CompletableFuture<CachedColumnRenderSource>[] adjacentLoadFutures = new CompletableFuture[4];
adjacentLoadFutures[0] = this.getRenderSourceForPosAsync(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.NORTH));
adjacentLoadFutures[1] = this.getRenderSourceForPosAsync(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.SOUTH));
adjacentLoadFutures[2] = this.getRenderSourceForPosAsync(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.EAST));
adjacentLoadFutures[3] = this.getRenderSourceForPosAsync(DhSectionPos.getAdjacentPos(this.pos, EDhDirection.WEST));
return CompletableFuture.allOf(adjacentLoadFutures).thenRun(() ->
{
try (CachedColumnRenderSource northRenderSource = adjacentLoadFutures[0].get();
CachedColumnRenderSource southRenderSource = adjacentLoadFutures[1].get();
CachedColumnRenderSource eastRenderSource = adjacentLoadFutures[2].get();
CachedColumnRenderSource westRenderSource = adjacentLoadFutures[3].get())
{
ColumnRenderSource[] adjacentRenderSections = new ColumnRenderSource[EDhDirection.ADJ_DIRECTIONS.length];
adjacentRenderSections[EDhDirection.NORTH.ordinal() - 2] = (northRenderSource != null) ? northRenderSource.columnRenderSource : null;
adjacentRenderSections[EDhDirection.SOUTH.ordinal() - 2] = (southRenderSource != null) ? southRenderSource.columnRenderSource : null;
adjacentRenderSections[EDhDirection.EAST.ordinal() - 2] = (eastRenderSource != null) ? eastRenderSource.columnRenderSource : null;
adjacentRenderSections[EDhDirection.WEST.ordinal() - 2] = (westRenderSource != null) ? westRenderSource.columnRenderSource : null;
boolean[] adjIsSameDetailLevel = new boolean[EDhDirection.ADJ_DIRECTIONS.length];
adjIsSameDetailLevel[EDhDirection.NORTH.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.NORTH);
adjIsSameDetailLevel[EDhDirection.SOUTH.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.SOUTH);
adjIsSameDetailLevel[EDhDirection.EAST.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.EAST);
adjIsSameDetailLevel[EDhDirection.WEST.ordinal() - 2] = this.isAdjacentPosSameDetailLevel(EDhDirection.WEST);
// the render sources are only needed by this synchronous method,
// then they can be closed
ColumnRenderBufferBuilder.makeLodRenderData(lodQuadBuilder, thisRenderSource, this.level, adjacentRenderSections, adjIsSameDetailLevel);
this.uploadToGpuAsync(lodQuadBuilder);
}
catch (Exception e)
{
LOGGER.error("Unexpected error while loading LodRenderSection [" + DhSectionPos.toString(this.pos) + "] adjacent data, Error: [" + e.getMessage() + "].", e);
}
finally
{
// can only be closed after the data has been processed and uploaded to the GPU
cachedRenderSource.close();
}
});
}
catch (Exception e)
{
LOGGER.error("Unexpected error while loading LodRenderSection ["+DhSectionPos.toString(this.pos)+"], Error: [" + e.getMessage() + "].", e);
return CompletableFuture.completedFuture(null);
}
});
}
@Nullable
private ColumnRenderSource getRenderSourceForPos(long pos)
/** async is done so each thread can run without waiting on others */
private CompletableFuture<CachedColumnRenderSource> getRenderSourceForPosAsync(long pos)
{
ReentrantLock lock = this.renderLoadLockContainer.getLockForPos(pos);
try
{
// we don't want multiple threads attempting to load the same position at the same time
// we don't want multiple threads attempting to load the same position at the same time,
// and we don't want to access the cache while invalidating it on a different thread
lock.lock();
// use the cached data if possible
ColumnRenderSource renderSource = this.cachedRenderSourceByPos.getIfPresent(pos);
if (renderSource != null)
CachedColumnRenderSource existingCachedRenderSource = this.cachedRenderSourceByPos.getIfPresent(pos);
if (existingCachedRenderSource != null)
{
return renderSource;
existingCachedRenderSource.markInUse();
return existingCachedRenderSource.loadFuture;
}
// generate new render source
try (FullDataSourceV2 fullDataSource = this.fullDataSourceProvider.get(pos))
PriorityTaskPicker.Executor executor = ThreadPoolUtil.getFileHandlerExecutor();
if (executor == null || executor.isTerminated())
{
renderSource = FullDataToRenderDataTransformer.transformFullDataToRenderSource(fullDataSource, this.level);
// only add valid data to the cache (to prevent null pointers)
if (renderSource != null)
// should only happen if the threadpool is actively being re-sized
return CompletableFuture.completedFuture(null);
}
// queue loading the render data
CompletableFuture<CachedColumnRenderSource> loadFuture = new CompletableFuture<>();
final CachedColumnRenderSource newCachedRenderSource = new CachedColumnRenderSource(loadFuture, lock, this.cachedRenderSourceByPos);
executor.execute(() ->
{
// generate new render source
try (FullDataSourceV2 fullDataSource = this.fullDataSourceProvider.get(pos))
{
this.cachedRenderSourceByPos.put(pos, renderSource);
newCachedRenderSource.columnRenderSource = FullDataToRenderDataTransformer.transformFullDataToRenderSource(fullDataSource, this.level);
}
}
catch (Exception e)
{
LOGGER.error("Unexpected issue creating render data for pos: ["+DhSectionPos.toString(pos)+"], error: ["+e.getMessage()+"].", e);
}
finally
{
loadFuture.complete(newCachedRenderSource);
}
});
this.cachedRenderSourceByPos.put(pos, newCachedRenderSource);
return renderSource;
return loadFuture;
}
catch (RejectedExecutionException ignore)
{
// the thread pool was probably shut down because it's size is being changed, just wait a sec and it should be back
return CompletableFuture.completedFuture(null);
}
catch (Exception e)
{
LOGGER.error("Unexpected issue getting and creating render data for pos: ["+DhSectionPos.toString(pos)+"], error: ["+e.getMessage()+"].", e);
return CompletableFuture.completedFuture(null);
}
finally
{
@@ -350,25 +424,24 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
public void setRenderingEnabled(boolean enabled) { this.renderingEnabled = enabled;}
/** @see LodRenderSection#setRenderingEnabled */
public void onRenderingEnabled() { this.level.loadBeaconBeamsInPos(this.pos); }
public void onRenderingEnabled() { this.startRenderingBeacons(); }
/** @see LodRenderSection#setRenderingEnabled */
public void onRenderingDisabled()
{
this.level.unloadBeaconBeamsInPos(this.pos);
this.stopRenderingBeacons();
if (Config.Client.Advanced.Debugging.DebugWireframe.showRenderSectionStatus.get())
{
// show that this position has just been disabled
DebugRenderer.makeParticle(
new DebugRenderer.BoxParticle(
new DebugRenderer.Box(this.pos, 128f, 156f, 0.09f, Color.CYAN.darker()),
0.2, 32f
)
new DebugRenderer.BoxParticle(
new DebugRenderer.Box(this.pos, 128f, 156f, 0.09f, Color.CYAN.darker()),
0.2, 32f
)
);
}
}
public boolean gpuUploadInProgress() { return this.getAndBuildRenderDataFuture != null; }
@@ -433,10 +506,17 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
if (this.fullDataSourceProvider.canRetrieveMissingDataSources() && this.fullDataSourceProvider.canQueueRetrieval())
{
// calculate the missing positions if not already done
if (this.missingGenerationPos == null)
if (this.missingGenerationPosFunc == null)
{
//this.missingGenerationPos = Suppliers.memoize(() -> this.fullDataSourceProvider.getPositionsToRetrieve(this.pos));
this.missingGenerationPos = Suppliers.memoizeWithExpiration(() -> this.fullDataSourceProvider.getPositionsToRetrieve(this.pos), 1, TimeUnit.MINUTES);
// TODO memoization is needed for multiplayer, otherwise
// new retrieval requests won't be submitted.
// TODO why is that the case? Shouldn't the missing positions be un-changing?
// TODO setting this value to low can cause world gen to slow down significantly
// due to a race condition where the world gen thinks it is finished, but the results
// haven't been saved to file yet, causing the gen to fire again
this.missingGenerationPosFunc = Suppliers.memoizeWithExpiration(
() -> this.fullDataSourceProvider.getPositionsToRetrieve(this.pos),
10, TimeUnit.MINUTES);
}
LongArrayList missingGenerationPos = this.getMissingGenerationPos();
@@ -465,6 +545,85 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
//=================//
// beacon handling //
//=================//
/** gets the active beacon list and stops/starts beacon rendering as necessary */
private void getAndRefreshRenderingBeacons()
{
// do nothing if beacon rendering or repos are unavailable
if (this.beaconBeamRepo == null
|| this.beaconRenderHandler == null)
{
return;
}
// Synchronized to prevent two threads for starting/stopping rendering at once
// Shouldn't be necessary, but just in case.
synchronized (this.activeBeaconList)
{
List<BeaconBeamDTO> activeBeacons = this.beaconBeamRepo.getAllBeamsForPos(this.pos);
// stop rendering current beacons
for (BeaconBeamDTO beam : this.activeBeaconList)
{
this.beaconRenderHandler.stopRenderingBeaconAtPos(beam.blockPos);
}
// swap old and new active beacon list
this.activeBeaconList.clear();
this.activeBeaconList.addAll(activeBeacons);
// start rendering new beacon list
for (BeaconBeamDTO beam : this.activeBeaconList)
{
this.beaconRenderHandler.startRenderingBeacon(beam);
}
}
}
private void stopRenderingBeacons()
{
// do nothing if beacon rendering is unavailable
if (this.beaconRenderHandler == null)
{
return;
}
synchronized (this.activeBeaconList)
{
for (BeaconBeamDTO beam : this.activeBeaconList)
{
this.beaconRenderHandler.stopRenderingBeaconAtPos(beam.blockPos);
}
}
}
private void startRenderingBeacons()
{
// do nothing if beacon rendering is unavailable
if (this.beaconRenderHandler == null)
{
return;
}
synchronized (this.activeBeaconList)
{
for (BeaconBeamDTO beam : this.activeBeaconList)
{
this.beaconRenderHandler.startRenderingBeacon(beam);
}
}
}
//==============//
// base methods //
//==============//
@@ -514,7 +673,7 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
}
this.level.unloadBeaconBeamsInPos(this.pos);
this.stopRenderingBeacons();
if (this.renderBuffer != null)
{
@@ -551,6 +710,7 @@ public class LodRenderSection implements IDebugRenderable, AutoCloseable
// while this should generally be a fast operation
// this is run on a separate thread to prevent lag on the render thread
executor.execute(() -> this.fullDataSourceProvider.removeRetrievalRequestIf((genPos) -> DhSectionPos.contains(this.pos, genPos)));
}
@@ -359,6 +359,7 @@ public class RenderBufferHandler implements AutoCloseable
// debug wireframe setup //
//=======================//
// TODO move this logic into LodRenderer so all the GL states can be handled there
boolean renderWireframe = Config.Client.Advanced.Debugging.renderWireframe.get();
if (renderWireframe)
{
@@ -25,10 +25,10 @@ import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.logging.ConfigBasedLogger;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.util.objects.GLMessage;
import com.seibel.distanthorizons.core.util.objects.GLMessageOutputStream;
import com.seibel.distanthorizons.core.util.objects.GLMessages.*;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftClientWrapper;
import com.seibel.distanthorizons.coreapi.ModInfo;
import com.seibel.distanthorizons.coreapi.util.StringUtil;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.lwjgl.glfw.GLFW;
@@ -38,31 +38,28 @@ import org.lwjgl.opengl.GLCapabilities;
import org.lwjgl.opengl.GLUtil;
import java.io.PrintStream;
import java.lang.invoke.MethodHandles;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
/**
* A singleton that holds references to different openGL contexts
* and GPU capabilities.
*
* <p>
* Helpful OpenGL resources:
* <p>
* https://www.seas.upenn.edu/~pcozzi/OpenGLInsights/OpenGLInsights-AsynchronousBufferTransfers.pdf <br>
* https://learnopengl.com/Advanced-OpenGL/Advanced-Data <br>
* https://www.slideshare.net/CassEveritt/approaching-zero-driver-overhead <br><br>
*
* https://gamedev.stackexchange.com/questions/91995/edit-vbo-data-or-create-a-new-one <br>
* https://stackoverflow.com/questions/63509735/massive-performance-loss-with-glmapbuffer <br><br>
*/
public class GLProxy
{
private static final IMinecraftClientWrapper MC = SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class);
private static final Logger LOGGER = DhLoggerBuilder.getLogger(MethodHandles.lookup().lookupClass().getSimpleName());
private static final Logger LOGGER = DhLoggerBuilder.getLogger();
public static final ConfigBasedLogger GL_LOGGER = new ConfigBasedLogger(LogManager.getLogger(GLProxy.class),
() -> Config.Common.Logging.logRendererGLEvent.get());
public static final Set<String> LOGGED_GL_MESSAGES = Collections.newSetFromMap(new ConcurrentHashMap<String, Boolean>());
private static GLProxy instance = null;
@@ -79,7 +76,29 @@ public class GLProxy
private final EDhApiGpuUploadMethod preferredUploadMethod;
public final GLMessage.Builder vanillaDebugMessageBuilder = GLMessage.Builder.DEFAULT_MESSAGE_BUILDER;
public final GLMessageBuilder vanillaDebugMessageBuilder =
new GLMessageBuilder(
(type) ->
{
if (type == EGLMessageType.POP_GROUP)
return false;
else if (type == EGLMessageType.PUSH_GROUP)
return false;
else if (type == EGLMessageType.MARKER)
return false;
else
return true;
},
(severity) ->
{
// notifications can generally be ignored (if they are logged at all)
if (severity == EGLMessageSeverity.NOTIFICATION)
return false;
else
return true;
},
null
);
@@ -258,6 +277,7 @@ public class GLProxy
// logging //
//=========//
/** this method is called on the render thread at the point of the GL Error */
private static void logMessage(GLMessage msg)
{
EDhApiGLErrorHandlingMode errorHandlingMode = Config.Client.Advanced.Debugging.OpenGl.glErrorHandlingMode.get();
@@ -268,44 +288,56 @@ public class GLProxy
if (msg.type == GLMessage.EType.ERROR || msg.type == GLMessage.EType.UNDEFINED_BEHAVIOR)
boolean onlyLogOnce = Config.Client.Advanced.Debugging.OpenGl.onlyLogGlErrorsOnce.get();
String errorMessage = "GL ERROR [" + msg.id + "] from [" + msg.source + "]: [" + msg.message + "]"+(onlyLogOnce ? " this message will only be logged once" : "")+".";
if (onlyLogOnce
&& !LOGGED_GL_MESSAGES.add(errorMessage))
{
// this message has already been logged
return;
}
// create an exception so we get a stacktrace of where the message was triggered from
RuntimeException exception = new RuntimeException(errorMessage);
if (msg.type == EGLMessageType.ERROR || msg.type == EGLMessageType.UNDEFINED_BEHAVIOR)
{
// critical error
GL_LOGGER.error("GL ERROR " + msg.id + " from " + msg.source + ": " + msg.message);
GL_LOGGER.error(exception.getMessage(), exception);
if (errorHandlingMode == EDhApiGLErrorHandlingMode.LOG_THROW)
{
throw new RuntimeException("GL ERROR: " + msg);
// will probably crash the game,
// good for quickly checking if there's a problem while preventing log spam
throw exception;
}
}
else
{
// non-critical log
GLMessage.ESeverity severity = msg.severity;
RuntimeException ex = new RuntimeException("GL MESSAGE: " + msg);
EGLMessageSeverity severity = msg.severity;
if (severity == null)
{
// just in case the message was malformed
severity = GLMessage.ESeverity.LOW;
severity = EGLMessageSeverity.LOW;
}
switch (severity)
{
case HIGH:
GL_LOGGER.error("{}", ex);
GL_LOGGER.error(exception.getMessage(), exception);
break;
case MEDIUM:
GL_LOGGER.warn("{}", ex);
GL_LOGGER.warn(exception.getMessage(), exception);
break;
case LOW:
GL_LOGGER.info("{}", ex);
GL_LOGGER.info(exception.getMessage(), exception);
break;
case NOTIFICATION:
GL_LOGGER.debug("{}", ex);
GL_LOGGER.debug(exception.getMessage(), exception);
break;
}
}
@@ -35,7 +35,7 @@ public class GLState
public int vao;
public int vbo;
public int ebo;
public int[] fbo;
public int fbo;
public int texture2D;
/** IE: GL_TEXTURE0, GL_TEXTURE1, etc. */
public int activeTextureNumber;
@@ -57,10 +57,10 @@ public class GLState
public boolean depth;
public boolean writeToDepthBuffer;
public int depthFunc;
//public boolean stencil;
//public int stencilFunc;
//public int stencilRef;
//public int stencilMask;
public boolean stencil;
public int stencilFunc;
public int stencilRef;
public int stencilMask;
public int[] view;
public boolean cull;
public int cullMode;
@@ -68,12 +68,7 @@ public class GLState
public GLState()
{
this.fbo = new int[FBO_MAX];
this.saveState();
}
public GLState() { this.saveState(); }
public void saveState()
{
@@ -82,7 +77,7 @@ public class GLState
this.vbo = GL32.glGetInteger(GL32.GL_ARRAY_BUFFER_BINDING);
this.ebo = GL32.glGetInteger(GL32.GL_ELEMENT_ARRAY_BUFFER_BINDING);
GL32.glGetIntegerv(GL32.GL_FRAMEBUFFER_BINDING, this.fbo);
this.fbo = GL32.glGetInteger(GL32.GL_FRAMEBUFFER_BINDING);
this.texture2D = GL32.glGetInteger(GL32.GL_TEXTURE_BINDING_2D);
this.activeTextureNumber = GL32.glGetInteger(GL32.GL_ACTIVE_TEXTURE);
@@ -101,9 +96,19 @@ public class GLState
GLMC.glActiveTexture(this.activeTextureNumber);
this.frameBufferTexture0 = GL32.glGetFramebufferAttachmentParameteri(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
this.frameBufferTexture1 = GL32.glGetFramebufferAttachmentParameteri(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT1, GL32.GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
this.frameBufferDepthTexture = GL32.glGetFramebufferAttachmentParameteri(GL32.GL_FRAMEBUFFER, GL32.GL_DEPTH_ATTACHMENT, GL32.GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
if (this.fbo != 0)
{
this.frameBufferTexture0 = GL32.glGetFramebufferAttachmentParameteri(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
this.frameBufferTexture1 = GL32.glGetFramebufferAttachmentParameteri(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT1, GL32.GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
this.frameBufferDepthTexture = GL32.glGetFramebufferAttachmentParameteri(GL32.GL_FRAMEBUFFER, GL32.GL_DEPTH_ATTACHMENT, GL32.GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME);
}
else
{
// attempting to get values from the default framebuffer can throw errors on Linux
this.frameBufferTexture0 = 0;
this.frameBufferTexture1 = 0;
this.frameBufferDepthTexture = 0;
}
this.blend = GL32.glIsEnabled(GL32.GL_BLEND);
this.scissor = GL32.glIsEnabled(GL32.GL_SCISSOR_TEST);
@@ -116,10 +121,10 @@ public class GLState
this.depth = GL32.glIsEnabled(GL32.GL_DEPTH_TEST);
this.writeToDepthBuffer = GL32.glGetInteger(GL32.GL_DEPTH_WRITEMASK) == GL32.GL_TRUE;
this.depthFunc = GL32.glGetInteger(GL32.GL_DEPTH_FUNC);
//this.stencil = GL32.glIsEnabled(GL32.GL_STENCIL_TEST);
//this.stencilFunc = GL32.glGetInteger(GL32.GL_STENCIL_FUNC);
//this.stencilRef = GL32.glGetInteger(GL32.GL_STENCIL_REF);
//this.stencilMask = GL32.glGetInteger(GL32.GL_STENCIL_VALUE_MASK);
this.stencil = GL32.glIsEnabled(GL32.GL_STENCIL_TEST);
this.stencilFunc = GL32.glGetInteger(GL32.GL_STENCIL_FUNC);
this.stencilRef = GL32.glGetInteger(GL32.GL_STENCIL_REF);
this.stencilMask = GL32.glGetInteger(GL32.GL_STENCIL_VALUE_MASK);
this.view = new int[4];
GL32.glGetIntegerv(GL32.GL_VIEWPORT, this.view);
this.cull = GL32.glIsEnabled(GL32.GL_CULL_FACE);
@@ -131,38 +136,33 @@ public class GLState
public String toString()
{
return "GLState{" +
"program=" + this.program + ", vao=" + this.vao + ", vbo=" + this.vbo + ", ebo=" + this.ebo + ", fbo=" + this.fbo[0] +
"program=" + this.program + ", vao=" + this.vao + ", vbo=" + this.vbo + ", ebo=" + this.ebo + ", fbo=" + this.fbo +
", text=" + GLEnums.getString(this.texture2D) + "@" + this.activeTextureNumber + ", text0=" + GLEnums.getString(this.texture0) +
", FB text0=" + this.frameBufferTexture0 +
", FB text1=" + this.frameBufferTexture1 +
", FB depth=" + this.frameBufferDepthTexture +
", blend=" + this.blend + ", scissor=" + this.scissor + ", blendMode=" + GLEnums.getString(this.blendSrcColor) + "," + GLEnums.getString(this.blendDstColor) +
", depth=" + this.depth +
//", depthFunc=" + GLEnums.getString(this.depthFunc) + ", stencil=" + this.stencil + ", stencilFunc=" +
//GLEnums.getString(this.stencilFunc) + ", stencilRef=" + this.stencilRef + ", stencilMask=" + this.stencilMask +
", depthFunc=" + GLEnums.getString(this.depthFunc) + ", stencil=" + this.stencil +
", stencilFunc=" + GLEnums.getString(this.stencilFunc) + ", stencilRef=" + this.stencilRef + ", stencilMask=" + this.stencilMask +
", view={x:" + this.view[0] + ", y:" + this.view[1] +
", w:" + this.view[2] + ", h:" + this.view[3] + "}" + ", cull=" + this.cull + ", cullMode="
+ GLEnums.getString(this.cullMode) + ", polyMode=" + GLEnums.getString(this.polyMode) +
", w:" + this.view[2] + ", h:" + this.view[3] + "}" + ", cull=" + this.cull +
", cullMode=" + GLEnums.getString(this.cullMode) + ", polyMode=" + GLEnums.getString(this.polyMode) +
'}';
}
public void RestoreFrameBuffer()
{
// explicitly unbinding the frame buffer is necessary to prevent GL_CLEAR calls from hitting the wrong buffer
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, 0);
for (int i = 0; i < FBO_MAX; i++)
{
int buffer = this.fbo[i];
if (i > 0 && buffer == 0) break;
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, GL32.glIsFramebuffer(buffer) ? buffer : 0);
}
}
public void restore()
{
this.RestoreFrameBuffer();
// explicitly unbinding the frame buffer is necessary to prevent GL_CLEAR calls from hitting the wrong buffer
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, 0);
boolean frameBufferSet = false;
if (this.fbo != 0 && GL32.glIsFramebuffer(this.fbo))
{
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, this.fbo);
frameBufferSet = true;
}
if (this.blend)
{
@@ -197,9 +197,13 @@ public class GLState
GLMC.glActiveTexture(this.activeTextureNumber);
GLMC.glBindTexture(GL32.glIsTexture(this.texture2D) ? this.texture2D : 0);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_TEXTURE_2D, this.frameBufferTexture0, 0);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT1, GL32.GL_TEXTURE_2D, this.frameBufferTexture1, 0);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_DEPTH_ATTACHMENT, GL32.GL_TEXTURE_2D, this.frameBufferDepthTexture, 0);
// attempting to set textures on the default frame buffer (ID 0) will throw errors
if (frameBufferSet)
{
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_TEXTURE_2D, this.frameBufferTexture0, 0);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT1, GL32.GL_TEXTURE_2D, this.frameBufferTexture1, 0);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_DEPTH_ATTACHMENT, GL32.GL_TEXTURE_2D, this.frameBufferDepthTexture, 0);
}
GL32.glBindVertexArray(GL32.glIsVertexArray(this.vao) ? this.vao : 0);
GL32.glBindBuffer(GL32.GL_ARRAY_BUFFER, GL32.glIsBuffer(this.vbo) ? this.vbo : 0);
@@ -229,15 +233,15 @@ public class GLState
}
GLMC.glDepthFunc(this.depthFunc);
//if (this.stencil)
//{
// GL32.glEnable(GL32.GL_STENCIL_TEST);
//}
//else
//{
// GL32.glDisable(GL32.GL_STENCIL_TEST);
//}
//GL32.glStencilFunc(this.stencilFunc, this.stencilRef, this.stencilMask);
if (this.stencil)
{
GL32.glEnable(GL32.GL_STENCIL_TEST);
}
else
{
GL32.glDisable(GL32.GL_STENCIL_TEST);
}
GL32.glStencilFunc(this.stencilFunc, this.stencilRef, this.stencilMask);
GL32.glViewport(this.view[0], this.view[1], this.view[2], this.view[3]);
if (this.cull)
@@ -223,7 +223,7 @@ public class GLBuffer implements AutoCloseable
this.destroyAsync();
this.create(true);
this.bind();
GL44.glBufferStorage(this.getBufferBindingTarget(), bb, bufferStorageHint);
GL44.glBufferStorage(this.getBufferBindingTarget(), bb, 0);
this.size = bbSize;
}
/** Requires the buffer to be bound */
@@ -25,22 +25,31 @@ import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.nio.ByteBuffer;
import com.seibel.distanthorizons.core.render.glObject.GLProxy;
import org.lwjgl.PointerBuffer;
import org.lwjgl.opengl.GL32;
import org.lwjgl.opengl.GL32C;
import org.lwjgl.system.MemoryStack;
import org.lwjgl.system.MemoryUtil;
import org.lwjgl.system.NativeType;
/**
* This object holds a OpenGL reference to a shader
* and allows for reading in and compiling a shader file.
*
* @author James Seibel
* @version 11-8-2021
*/
public class Shader
{
/** OpenGL shader ID */
public final int id;
//==============//
// constructors //
//==============//
/**
* Creates a shader with specified type.
*
@@ -51,52 +60,99 @@ public class Shader
*/
public Shader(int type, String path, boolean absoluteFilePath)
{
GLProxy.GL_LOGGER.info("Loading shader at " + path);
GLProxy.GL_LOGGER.info("Loading shader at [" + path + "]");
// Create an empty shader object
id = GL32.glCreateShader(type);
StringBuilder source = loadFile(path, absoluteFilePath, new StringBuilder());
GL32.glShaderSource(id, source);
this.id = GL32.glCreateShader(type);
if (this.id == 0)
{
throw new IllegalArgumentException("Failed to create shader with type ["+type+"].");
}
GL32.glCompileShader(id);
StringBuilder source = loadFile(path, absoluteFilePath, new StringBuilder());
safeShaderSource(this.id, source);
GL32.glCompileShader(this.id);
// check if the shader compiled
int status = GL32.glGetShaderi(id, GL32.GL_COMPILE_STATUS);
int status = GL32.glGetShaderi(this.id, GL32.GL_COMPILE_STATUS);
if (status != GL32.GL_TRUE)
{
String message = "Shader compiler error. Details: " + GL32.glGetShaderInfoLog(id);
free(); // important!
String message = "Shader compiler error. Details: ["+GL32.glGetShaderInfoLog(this.id)+"].";
this.free(); // important!
throw new RuntimeException(message);
}
GLProxy.GL_LOGGER.info("Shader at " + path + " loaded sucessfully.");
GLProxy.GL_LOGGER.info("Shader at " + path + " loaded successfully.");
}
public Shader(int type, String sourceString)
{
GLProxy.GL_LOGGER.info("Loading shader with type: {}", type);
GLProxy.GL_LOGGER.debug("Source:\n{}", sourceString);
// Create an empty shader object
id = GL32.glCreateShader(type);
GL32.glShaderSource(id, sourceString);
GLProxy.GL_LOGGER.info("Loading shader with type: ["+type+"]");
GLProxy.GL_LOGGER.debug("Source: \n["+sourceString+"]");
if (sourceString == null || sourceString.isEmpty())
{
throw new IllegalArgumentException("No shader source given.");
}
GL32.glCompileShader(id);
// Create an empty shader object
this.id = GL32.glCreateShader(type);
if (this.id == 0)
{
throw new IllegalArgumentException("Failed to create shader with type ["+type+"] and Source: \n["+sourceString+"].");
}
safeShaderSource(this.id, sourceString);
GL32.glCompileShader(this.id);
// check if the shader compiled
int status = GL32.glGetShaderi(id, GL32.GL_COMPILE_STATUS);
int status = GL32.glGetShaderi(this.id, GL32.GL_COMPILE_STATUS);
if (status != GL32.GL_TRUE)
{
String message = "Shader compiler error. Details: " + GL32.glGetShaderInfoLog(id);
message += "\nSource:\n" + sourceString;
free(); // important!
String message = "Shader compiler error. Details: [" + GL32.glGetShaderInfoLog(this.id) + "]\n";
message += "Source: \n[" + sourceString + "]";
this.free(); // important!
throw new RuntimeException(message);
}
GLProxy.GL_LOGGER.info("Shader loaded sucessfully.");
}
// REMEMBER to always free the resource!
public void free()
//=========//
// helpers //
//=========//
/**
* Identical in function to {@link GL32C#glShaderSource(int, CharSequence)} but
* passes a null pointer for string length to force the driver to rely on the null
* terminator for string length. This is a workaround for an apparent flaw with some
* AMD drivers that don't receive or interpret the length correctly, resulting in
* an access violation when the driver tries to read past the string memory.
*
* <p>Hat tip to fewizz for the find and the fix.
*
* <p>Source: https://github.com/vram-guild/canvas/commit/820bf754092ccaf8d0c169620c2ff575722d7d96
*/
private static void safeShaderSource(@NativeType("GLuint") int glId, @NativeType("GLchar const **") CharSequence source)
{
GL32.glDeleteShader(id);
final MemoryStack stack = MemoryStack.stackGet();
final int stackPointer = stack.getPointer();
try
{
final ByteBuffer sourceBuffer = MemoryUtil.memUTF8(source, true);
final PointerBuffer pointers = stack.mallocPointer(1);
pointers.put(sourceBuffer);
GL32.nglShaderSource(glId, 1, pointers.address0(), 0);
org.lwjgl.system.APIUtil.apiArrayFree(pointers.address0(), 1);
}
finally
{
stack.setPointer(stackPointer);
}
}
public void free() { GL32.glDeleteShader(this.id); }
public static StringBuilder loadFile(String path, boolean absoluteFilePath, StringBuilder stringBuilder)
{
try
@@ -132,4 +188,6 @@ public class Shader
return stringBuilder;
}
}
@@ -1,5 +1,7 @@
package com.seibel.distanthorizons.core.render.glObject.texture;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftGLWrapper;
import org.lwjgl.opengl.GL11C;
import org.lwjgl.opengl.GL13C;
import org.lwjgl.opengl.GL43C;
@@ -8,12 +10,15 @@ import java.nio.ByteBuffer;
public class DHDepthTexture
{
private static final IMinecraftGLWrapper GLMC = SingletonInjector.INSTANCE.get(IMinecraftGLWrapper.class);
private int id;
public DHDepthTexture(int width, int height, EDhDepthBufferFormat format)
{
this.id = GL43C.glGenTextures();
resize(width, height, format);
this.resize(width, height, format);
GL43C.glTexParameteri(GL11C.GL_TEXTURE_2D, GL11C.GL_TEXTURE_MIN_FILTER, GL11C.GL_NEAREST);
GL43C.glTexParameteri(GL11C.GL_TEXTURE_2D, GL11C.GL_TEXTURE_MAG_FILTER, GL11C.GL_NEAREST);
@@ -24,27 +29,31 @@ public class DHDepthTexture
}
// For internal use by Iris for copying data. Do not use this in DH.
public DHDepthTexture(int id) {
this.id = id;
}
public DHDepthTexture(int id) { this.id = id; }
public void resize(int width, int height, EDhDepthBufferFormat format)
{
GL43C.glBindTexture(GL43C.GL_TEXTURE_2D, getTextureId());
GL43C.glBindTexture(GL43C.GL_TEXTURE_2D, this.getTextureId());
GL43C.glTexImage2D(GL11C.GL_TEXTURE_2D, 0, format.getGlInternalFormat(), width, height, 0,
format.getGlType(), format.getGlFormat(), (ByteBuffer) null);
}
public int getTextureId()
{
if (id == -1) throw new IllegalStateException("Depth texture does not exist!");
return id;
if (this.id == -1)
{
throw new IllegalStateException("Depth texture does not exist!");
}
return this.id;
}
public void destroy()
{
GL43C.glDeleteTextures(getTextureId());
GLMC.glDeleteTextures(this.getTextureId());
this.id = -1;
}
}
@@ -1,5 +1,7 @@
package com.seibel.distanthorizons.core.render.glObject.texture;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftGLWrapper;
import org.joml.Vector2i;
import org.lwjgl.opengl.GL11C;
import org.lwjgl.opengl.GL13C;
@@ -9,6 +11,9 @@ import java.nio.ByteBuffer;
public class DhColorTexture
{
private static final IMinecraftGLWrapper GLMC = SingletonInjector.INSTANCE.get(IMinecraftGLWrapper.class);
private final EDhInternalTextureFormat internalFormat;
private final EDhPixelFormat format;
private final EDhPixelType type;
@@ -100,7 +105,7 @@ public class DhColorTexture
this.throwIfInvalid();
this.isValid = false;
GL43C.glDeleteTextures(this.id);
GLMC.glDeleteTextures(this.id);
}
/** @throws IllegalStateException if the texture isn't valid */
@@ -86,22 +86,31 @@ public class FadeRenderer
this.fadeFramebuffer = -1;
}
if (this.fadeTexture != -1)
{
GLMC.glDeleteTextures(this.fadeTexture);
this.fadeTexture = -1;
}
this.fadeFramebuffer = GL32.glGenFramebuffers();
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, this.fadeFramebuffer);
this.fadeTexture = GL32.glGenTextures();
GLMC.glBindTexture(this.fadeTexture);
GL32.glTexImage2D(GL32.GL_TEXTURE_2D, 0, GL32.GL_RGBA16, width, height, 0, GL32.GL_RGBA, GL32.GL_UNSIGNED_SHORT_4_4_4_4, (ByteBuffer) null);
GL32.glTexParameteri(GL32.GL_TEXTURE_2D, GL32.GL_TEXTURE_MIN_FILTER, GL32.GL_LINEAR);
GL32.glTexParameteri(GL32.GL_TEXTURE_2D, GL32.GL_TEXTURE_MAG_FILTER, GL32.GL_LINEAR);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_TEXTURE_2D, this.fadeTexture, 0);
// Applying the fade texture is only needed if MC is drawing to their own frame buffer,
// otherwise we can directly render to their texture
if (MC_RENDER.mcRendersToFrameBuffer())
{
if (this.fadeTexture != -1)
{
GLMC.glDeleteTextures(this.fadeTexture);
this.fadeTexture = -1;
}
this.fadeTexture = GL32.glGenTextures();
GLMC.glBindTexture(this.fadeTexture);
GL32.glTexImage2D(GL32.GL_TEXTURE_2D, 0, GL32.GL_RGBA16, width, height, 0, GL32.GL_RGBA, GL32.GL_UNSIGNED_SHORT_4_4_4_4, (ByteBuffer) null);
GL32.glTexParameteri(GL32.GL_TEXTURE_2D, GL32.GL_TEXTURE_MIN_FILTER, GL32.GL_LINEAR);
GL32.glTexParameteri(GL32.GL_TEXTURE_2D, GL32.GL_TEXTURE_MAG_FILTER, GL32.GL_LINEAR);
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_TEXTURE_2D, this.fadeTexture, 0);
}
else
{
GL32.glFramebufferTexture2D(GL32.GL_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, GL32.GL_TEXTURE_2D, MC_RENDER.getColorTextureId(), 0);
}
}
@@ -146,8 +155,13 @@ public class FadeRenderer
profiler.popPush("Fade Apply");
FadeApplyShader.INSTANCE.fadeTexture = this.fadeTexture;
FadeApplyShader.INSTANCE.render(partialTicks);
// Applying the fade texture is only needed if MC is drawing to their own frame buffer,
// otherwise we can directly render to their texture
if (MC_RENDER.mcRendersToFrameBuffer())
{
FadeApplyShader.INSTANCE.fadeTexture = this.fadeTexture;
FadeApplyShader.INSTANCE.render(partialTicks);
}
profiler.pop();
}
@@ -23,6 +23,7 @@ import com.seibel.distanthorizons.api.interfaces.override.rendering.IDhApiFrameb
import com.seibel.distanthorizons.api.interfaces.override.rendering.IDhApiShaderProgram;
import com.seibel.distanthorizons.api.methods.events.abstractEvents.*;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiRenderParam;
import com.seibel.distanthorizons.api.methods.events.sharedParameterObjects.DhApiTextureCreatedParam;
import com.seibel.distanthorizons.core.config.Config;
import com.seibel.distanthorizons.core.dataObjects.render.bufferBuilding.ColumnRenderBuffer;
import com.seibel.distanthorizons.core.dependencyInjection.ModAccessorInjector;
@@ -557,34 +558,24 @@ public class LodRenderer
activeFrameBuffer.bind();
boolean clearTextures = !ApiEventInjector.INSTANCE.fireAllEvents(DhApiBeforeTextureClearEvent.class, renderEventParam);
if (clearTextures)
{
if (this.usingMcFrameBuffer && framebufferOverride == null)
{
// Due to using MC/Optifine's framebuffer we need to re-bind the depth texture,
// otherwise we'll be writing to MC/Optifine's depth texture which causes rendering issues
activeFrameBuffer.addDepthAttachment(this.depthTexture.getTextureId(), EDhDepthBufferFormat.DEPTH32F.isCombinedStencil());
// don't clear the color texture, that removes the sky
GL32.glClear(GL32.GL_DEPTH_BUFFER_BIT);
}
else if (firstPass)
{
GL32.glClear(GL32.GL_COLOR_BUFFER_BIT | GL32.GL_DEPTH_BUFFER_BIT);
}
}
// by default draw everything as triangles
GL32.glPolygonMode(GL32.GL_FRONT_AND_BACK, GL32.GL_FILL);
GLMC.enableFaceCulling();
GLMC.glBlendFunc(GL32.GL_SRC_ALPHA, GL32.GL_ONE_MINUS_SRC_ALPHA);
GLMC.glBlendFuncSeparate(GL32.GL_SRC_ALPHA, GL32.GL_ONE_MINUS_SRC_ALPHA, GL32.GL_ONE, GL32.GL_ZERO);
GL32.glDisable(GL32.GL_SCISSOR_TEST);
// Enable depth test and depth mask
GLMC.enableDepthTest();
GLMC.glDepthFunc(GL32.GL_LESS);
GLMC.enableDepthMask();
// This is required for MC versions 1.21.5+
// due to MC updating the lightmap by changing the viewport size
GL32.glViewport(0, 0, this.cachedWidth, this.cachedHeight);
/*---------Bind required objects--------*/
// Setup LodRenderProgram and the LightmapTexture if it has not yet been done
// also binds LightmapTexture, VAO, and ShaderProgram
@@ -605,6 +596,33 @@ public class LodRenderer
}
this.lodRenderProgram.fillUniformData(renderEventParam);
// needs to be fired after all the textures have been created/bound
boolean clearTextures = !ApiEventInjector.INSTANCE.fireAllEvents(DhApiBeforeTextureClearEvent.class, renderEventParam);
if (clearTextures)
{
GL32.glClearDepth(1.0);
float[] clearColorValues = new float[4];
GL32.glGetFloatv(GL32.GL_COLOR_CLEAR_VALUE, clearColorValues);
GL32.glClearColor(clearColorValues[0], clearColorValues[1], clearColorValues[2], 1.0f);
if (this.usingMcFrameBuffer && framebufferOverride == null)
{
// Due to using MC/Optifine's framebuffer we need to re-bind the depth texture,
// otherwise we'll be writing to MC/Optifine's depth texture which causes rendering issues
activeFrameBuffer.addDepthAttachment(this.depthTexture.getTextureId(), EDhDepthBufferFormat.DEPTH32F.isCombinedStencil());
// don't clear the color texture, that removes the sky
GL32.glClear(GL32.GL_DEPTH_BUFFER_BIT);
}
else if (firstPass)
{
GL32.glClear(GL32.GL_COLOR_BUFFER_BIT | GL32.GL_DEPTH_BUFFER_BIT);
}
}
}
/** Setup all render objects - MUST be called on the render thread */
@@ -656,7 +674,7 @@ public class LodRenderer
if(this.framebuffer.getStatus() != GL32.GL_FRAMEBUFFER_COMPLETE)
{
// This generally means something wasn't bound, IE missing either the color or depth texture
SPAM_LOGGER.warn("FrameBuffer ["+this.framebuffer.getId()+"] isn't complete.");
EVENT_LOGGER.warn("FrameBuffer ["+this.framebuffer.getId()+"] isn't complete.");
}
@@ -675,12 +693,15 @@ public class LodRenderer
this.cachedWidth = MC_RENDER.getTargetFrameBufferViewportWidth();
this.cachedHeight = MC_RENDER.getTargetFrameBufferViewportHeight();
DhApiTextureCreatedParam textureCreatedParam = new DhApiTextureCreatedParam(
oldWidth, oldHeight,
this.cachedWidth, this.cachedHeight
);
ApiEventInjector.INSTANCE.fireAllEvents(DhApiColorDepthTextureCreatedEvent.class,
new DhApiColorDepthTextureCreatedEvent.EventParam(
oldWidth, oldHeight,
this.cachedWidth, this.cachedHeight
));
ApiEventInjector.INSTANCE.fireAllEvents(DhApiColorDepthTextureCreatedEvent.class, new DhApiColorDepthTextureCreatedEvent.EventParam(textureCreatedParam));
ApiEventInjector.INSTANCE.fireAllEvents(DhApiBeforeColorDepthTextureCreatedEvent.class, textureCreatedParam);
// also update the override if present
@@ -712,26 +733,10 @@ public class LodRenderer
{
this.nullableColorTexture = null;
}
}
private Color getFogColor(float partialTicks)
{
Color fogColor;
if (Config.Client.Advanced.Graphics.Fog.colorMode.get() == EDhApiFogColorMode.USE_SKY_COLOR)
{
fogColor = MC_RENDER.getSkyColor();
}
else
{
fogColor = MC_RENDER.getFogColor(partialTicks);
}
return fogColor;
ApiEventInjector.INSTANCE.fireAllEvents(DhApiAfterColorDepthTextureCreatedEvent.class, textureCreatedParam);
}
private Color getSpecialFogColor(float partialTicks) { return MC_RENDER.getSpecialFogColor(partialTicks); }
@@ -748,7 +753,10 @@ public class LodRenderer
public static int getActiveColorTextureId() { return activeColorTextureId; }
private void setActiveDepthTextureId(int depthTextureId) { activeDepthTextureId = depthTextureId; }
/** Returns -1 if no texture has been bound yet */
/**
* FIXME it's possible for this to return an invalid texture ID if the renderer is being re-built at the same time
* Returns -1 if no texture has been bound yet
*/
public static int getActiveDepthTextureId() { return activeDepthTextureId; }
@@ -796,6 +804,9 @@ public class LodRenderer
if (this.depthTexture != null)
this.depthTexture.destroy();
this.setActiveDepthTextureId(-1);
this.setActiveColorTextureId(-1);
EVENT_LOGGER.info("Renderer Cleanup Complete");
});
}
@@ -31,6 +31,7 @@ import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.pos.blockPos.DhBlockPos;
import com.seibel.distanthorizons.core.sql.dto.BeaconBeamDTO;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.util.RenderUtil;
import com.seibel.distanthorizons.core.util.math.Vec3d;
import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftRenderWrapper;
@@ -50,8 +51,6 @@ public class BeaconRenderHandler
private static final Logger LOGGER = DhLoggerBuilder.getLogger();
private static final IMinecraftRenderWrapper MC_RENDER = SingletonInjector.INSTANCE.get(IMinecraftRenderWrapper.class);
private static final int BEAM_TOP_Y = 6_000;
/** how often should we check if a beacon should be culled? */
private static final int MAX_CULLING_FREQUENCY_IN_MS = 1_000;
@@ -98,9 +97,10 @@ public class BeaconRenderHandler
if (this.beaconBlockPosSet.add(beacon.blockPos))
{
int maxBeaconBeamHeight = Config.Client.Advanced.Graphics.GenericRendering.beaconRenderHeight.get();
DhApiRenderableBox beaconBox = new DhApiRenderableBox(
new DhApiVec3d(beacon.blockPos.getX(), beacon.blockPos.getY() + 1, beacon.blockPos.getZ()),
new DhApiVec3d(beacon.blockPos.getX() + 1, BEAM_TOP_Y, beacon.blockPos.getZ() + 1),
new DhApiVec3d(beacon.blockPos.getX() + 1, maxBeaconBeamHeight, beacon.blockPos.getZ() + 1),
beacon.color,
EDhApiBlockMaterial.ILLUMINATED
);
@@ -215,7 +215,7 @@ public class BeaconRenderHandler
double mcRenderDistance = MC_RENDER.getRenderDistance() * LodUtil.CHUNK_WIDTH;
// multiplying by overdraw prevention helps reduce beacons from rendering strangely
// on the border of DH's render distance
mcRenderDistance *= Config.Client.Advanced.Graphics.Culling.overdrawPrevention.get();
mcRenderDistance *= RenderUtil.getAutoOverdrawPrevention();
// Clear the existing box group so we can re-populate it.
@@ -75,6 +75,18 @@ public class DhApplyShader extends AbstractShaderRenderer
@Override
protected void onRender()
{
if (MC_RENDER.mcRendersToFrameBuffer())
{
this.renderToFrameBuffer();
}
else
{
this.renderToMcTexture();
}
}
// TODO merge duplicate code between these to render methods
private void renderToFrameBuffer()
{
int targetFrameBuffer = MC_RENDER.getTargetFrameBuffer();
if (targetFrameBuffer == -1)
@@ -87,9 +99,15 @@ public class DhApplyShader extends AbstractShaderRenderer
GLMC.disableDepthTest();
GLMC.enableBlend();
GL32.glBlendEquation(GL32.GL_FUNC_ADD);
GLMC.glBlendFunc(GL32.GL_ONE, GL32.GL_ONE_MINUS_SRC_ALPHA);
// blending isn't needed, we're manually merging the MC and DH textures
// Note: this prevents the sun/moon and stars from rendering through transparent LODs,
// however this also fixes transparent LODs from glowing when rendered against the sky during the day
GLMC.disableBlend();
// old blending logic in case it's ever needed:
//GLMC.enableBlend();
//GL32.glBlendEquation(GL32.GL_FUNC_ADD);
//GLMC.glBlendFunc(GL32.GL_ONE, GL32.GL_ONE_MINUS_SRC_ALPHA);
GLMC.glActiveTexture(GL32.GL_TEXTURE0);
GLMC.glBindTexture(LodRenderer.getActiveColorTextureId());
@@ -110,5 +128,66 @@ public class DhApplyShader extends AbstractShaderRenderer
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, targetFrameBuffer);
}
private void renderToMcTexture()
{
int targetColorTextureId = MC_RENDER.getColorTextureId();
if (targetColorTextureId == -1)
{
return;
}
int dhFrameBufferId = LodRenderer.getActiveFramebufferId();
if (dhFrameBufferId == -1)
{
return;
}
int mcFrameBufferId = MC_RENDER.getTargetFrameBuffer();
if (mcFrameBufferId == -1)
{
return;
}
GLState state = new GLState();
GLMC.disableDepthTest();
// blending isn't needed, we're just directly merging the MC and DH textures
// Note: this prevents the sun/moon and stars from rendering through transparent LODs,
// however this also fixes
GLMC.disableBlend();
// old blending logic in case it's ever needed:
//GLMC.enableBlend();
//GL32.glBlendEquation(GL32.GL_FUNC_ADD);
//GLMC.glBlendFunc(GL32.GL_ONE, GL32.GL_ONE_MINUS_SRC_ALPHA);
GLMC.glActiveTexture(GL32.GL_TEXTURE0);
GLMC.glBindTexture(LodRenderer.getActiveColorTextureId());
GL32.glUniform1i(this.gDhColorTextureUniform, 0);
GLMC.glActiveTexture(GL32.GL_TEXTURE1);
GLMC.glBindTexture(LodRenderer.getActiveDepthTextureId());
GL32.glUniform1i(this.gDepthMapUniform, 1);
GL32.glFramebufferTexture(GL32.GL_DRAW_FRAMEBUFFER, GL32.GL_COLOR_ATTACHMENT0, targetColorTextureId, 0);
// Copy to MC's texture via MC's framebuffer
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, dhFrameBufferId);
ScreenQuad.INSTANCE.render();
// restore everything, except at this point the MC framebuffer should now be used instead
state.restore();
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, mcFrameBufferId);
}
}
@@ -20,6 +20,7 @@
package com.seibel.distanthorizons.core.render.renderer.shaders;
import com.seibel.distanthorizons.core.dependencyInjection.SingletonInjector;
import com.seibel.distanthorizons.core.render.glObject.GLState;
import com.seibel.distanthorizons.core.render.glObject.shader.ShaderProgram;
import com.seibel.distanthorizons.core.render.renderer.FadeRenderer;
import com.seibel.distanthorizons.core.render.renderer.LodRenderer;
@@ -104,6 +105,12 @@ public class FadeApplyShader extends AbstractShaderRenderer
@Override
protected void onRender()
{
if (!MC_RENDER.mcRendersToFrameBuffer())
{
throw new IllegalStateException("If Minecraft is directly rendering to a texture the apply shader isn't needed, just draw the fade directly to the MC color texture.");
}
GLMC.disableBlend();
// Depth testing must be disabled otherwise this application shader won't apply anything.
@@ -119,6 +126,9 @@ public class FadeApplyShader extends AbstractShaderRenderer
ScreenQuad.INSTANCE.render();
GLMC.enableDepthTest();
}
}
@@ -155,6 +155,19 @@ public class FadeShader extends AbstractShaderRenderer
@Override
protected void onRender()
{
int depthTextureId = LodRenderer.getActiveDepthTextureId();
int colorTextureId = LodRenderer.getActiveColorTextureId();
if (depthTextureId == -1
|| colorTextureId == -1)
{
// the renderer is currently being re-built and/or inactive,
// we don't need to/can't render fading
return;
}
GLMC.glBindFramebuffer(GL32.GL_FRAMEBUFFER, this.frameBuffer);
GLMC.disableScissorTest();
GLMC.disableDepthTest();
@@ -165,7 +178,7 @@ public class FadeShader extends AbstractShaderRenderer
GL32.glUniform1i(this.uMcDepthTexture, 0);
GLMC.glActiveTexture(GL32.GL_TEXTURE1);
GLMC.glBindTexture(LodRenderer.getActiveDepthTextureId());
GLMC.glBindTexture(depthTextureId);
GL32.glUniform1i(this.uDhDepthTexture, 1);
GLMC.glActiveTexture(GL32.GL_TEXTURE2);
@@ -173,7 +186,7 @@ public class FadeShader extends AbstractShaderRenderer
GL32.glUniform1i(this.uCombinedMcDhColorTexture, 2);
GLMC.glActiveTexture(GL32.GL_TEXTURE3);
GLMC.glBindTexture(LodRenderer.getActiveColorTextureId());
GLMC.glBindTexture(colorTextureId);
GL32.glUniform1i(this.uDhColorTexture, 3);
@@ -84,6 +84,7 @@ public class FogApplyShader extends AbstractShaderRenderer
GLMC.glActiveTexture(GL32.GL_TEXTURE1);
GLMC.glBindTexture(LodRenderer.getActiveDepthTextureId());
GL32.glUniform1i(this.depthTextureUniform, 1);
}
@@ -28,9 +28,11 @@ import com.seibel.distanthorizons.core.render.glObject.shader.ShaderProgram;
import com.seibel.distanthorizons.core.render.renderer.LodRenderer;
import com.seibel.distanthorizons.core.render.renderer.ScreenQuad;
import com.seibel.distanthorizons.core.util.LodUtil;
import com.seibel.distanthorizons.core.wrapperInterfaces.IVersionConstants;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftClientWrapper;
import com.seibel.distanthorizons.core.util.math.Mat4f;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftGLWrapper;
import com.seibel.distanthorizons.core.wrapperInterfaces.minecraft.IMinecraftRenderWrapper;
import org.lwjgl.opengl.GL32;
import java.awt.*;
@@ -41,11 +43,14 @@ public class FogShader extends AbstractShaderRenderer
private static final IMinecraftClientWrapper MC = SingletonInjector.INSTANCE.get(IMinecraftClientWrapper.class);
private static final IMinecraftGLWrapper GLMC = SingletonInjector.INSTANCE.get(IMinecraftGLWrapper.class);
private static final IMinecraftRenderWrapper MC_RENDER = SingletonInjector.INSTANCE.get(IMinecraftRenderWrapper.class);
public int frameBuffer;
private Mat4f inverseMvmProjMatrix;
private Mat4f inverseMvmProjMatrix;
//==========//
@@ -253,7 +258,15 @@ public class FogShader extends AbstractShaderRenderer
// this is necessary for MC 1.16 (IE Legacy OpenGL)
// otherwise the framebuffer isn't cleared correctly and the fog smears across the screen
GL32.glClear(GL32.GL_COLOR_BUFFER_BIT | GL32.GL_DEPTH_BUFFER_BIT);
if (MC_RENDER.runningLegacyOpenGL())
{
// in another part of the DH code we set the fog color to opaque, here it needs to be transparent
float[] clearColorValues = new float[4];
GL32.glGetFloatv(GL32.GL_COLOR_CLEAR_VALUE, clearColorValues);
GL32.glClearColor(clearColorValues[0], clearColorValues[1], clearColorValues[2], 0.0f);
GL32.glClear(GL32.GL_COLOR_BUFFER_BIT | GL32.GL_DEPTH_BUFFER_BIT);
}
ScreenQuad.INSTANCE.render();
@@ -134,9 +134,6 @@ public class SSAOApplyShader extends AbstractShaderRenderer
// it should be automatically restored after rendering is complete.
GLMC.disableDepthTest();
GLMC.glActiveTexture(GL32.GL_TEXTURE0);
GLMC.glBindTexture(0);
// apply the rendered SSAO to the LODs
GLMC.glBindFramebuffer(GL32.GL_READ_FRAMEBUFFER, SSAOShader.INSTANCE.frameBuffer);
GLMC.glBindFramebuffer(GL32.GL_DRAW_FRAMEBUFFER, LodRenderer.getActiveFramebufferId());
@@ -16,11 +16,14 @@ public class DbConnectionClosedException extends SQLException
// helper methods //
public static boolean IsClosedException(SQLException e)
public static boolean isClosedException(SQLException e)
{
// TODO long term we should prevent using repos that are closed, but for now this is the easier solution
String message = e.getMessage().toLowerCase();
return message.contains("connection closed") || message.contains("pointer is closed") || message.contains("database has been closed");
return message.contains("connection closed")
|| message.contains("pointer is closed")
|| message.contains("stmt pointer is closed")
|| message.contains("database has been closed");
}
}
@@ -29,6 +29,7 @@ import com.seibel.distanthorizons.core.pooling.PhantomArrayListParent;
import com.seibel.distanthorizons.core.pooling.PhantomArrayListPool;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.network.INetworkObject;
import com.seibel.distanthorizons.core.util.BoolUtil;
import com.seibel.distanthorizons.core.util.FullDataPointUtil;
import com.seibel.distanthorizons.core.util.ListUtil;
import com.seibel.distanthorizons.core.util.LodUtil;
@@ -40,6 +41,7 @@ import io.netty.buffer.ByteBuf;
import it.unimi.dsi.fastutil.bytes.ByteArrayList;
import it.unimi.dsi.fastutil.longs.LongArrayList;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import java.io.*;
@@ -70,7 +72,12 @@ public class FullDataSourceV2DTO
public byte dataFormatVersion;
public byte compressionModeValue;
public boolean applyToParent;
/** Will be null if we don't want to update this value in the DB */
@Nullable
public Boolean applyToParent;
/** Will be null if we don't want to update this value in the DB */
@Nullable
public Boolean applyToChildren;
public long lastModifiedUnixDateTime;
public long createdUnixDateTime;
@@ -97,12 +104,15 @@ public class FullDataSourceV2DTO
// populate individual variables
{
dto.pos = dataSource.getPos();
dto.dataChecksum = (dataSource.mapping.hashCode() * 4217) + dataSource.hashCode();
// the mapping hash isn't included since it takes significantly longer to calculate and
// as of the time of this comment (2025-1-22) the checksum isn't used for anything so changing it shouldn't cause any issues
dto.dataChecksum = dataSource.hashCode();
dto.dataFormatVersion = FullDataSourceV2.DATA_FORMAT_VERSION;
dto.compressionModeValue = compressionModeEnum.value;
dto.lastModifiedUnixDateTime = dataSource.lastModifiedUnixDateTime;
dto.createdUnixDateTime = dataSource.createdUnixDateTime;
dto.applyToParent = dataSource.applyToParent;
dto.applyToChildren = dataSource.applyToChildren;
dto.levelMinY = dataSource.levelMinY;
}
@@ -121,8 +131,6 @@ public class FullDataSourceV2DTO
this.compressedColumnGenStepByteArray = this.pooledArraysCheckout.getByteArray(1, 0);
this.compressedWorldCompressionModeByteArray = this.pooledArraysCheckout.getByteArray(2, 0);
this.compressedMappingByteArray = this.pooledArraysCheckout.getByteArray(3, 0);
this.pooledArraysCheckout = null;
}
@@ -134,7 +142,16 @@ public class FullDataSourceV2DTO
public FullDataSourceV2 createDataSource(@NotNull ILevelWrapper levelWrapper) throws IOException, InterruptedException, DataCorruptedException
{
FullDataSourceV2 dataSource = FullDataSourceV2.createEmpty(this.pos);
this.internalPopulateDataSource(dataSource, levelWrapper, false);
try
{
this.internalPopulateDataSource(dataSource, levelWrapper, false);
}
catch (Exception e)
{
dataSource.close();
throw e;
}
return dataSource;
}
@@ -195,6 +212,15 @@ public class FullDataSourceV2DTO
dataSource.isEmpty = false;
if (this.applyToParent != null)
{
dataSource.applyToParent = this.applyToParent;
}
if (this.applyToChildren != null)
{
dataSource.applyToChildren = this.applyToChildren;
}
return dataSource;
}
@@ -379,7 +405,8 @@ public class FullDataSourceV2DTO
out.writeByte(this.dataFormatVersion);
out.writeByte(this.compressionModeValue);
out.writeBoolean(this.applyToParent);
out.writeBoolean(BoolUtil.falseIfNull(this.applyToParent));
out.writeBoolean(BoolUtil.falseIfNull(this.applyToChildren));
out.writeLong(this.lastModifiedUnixDateTime);
out.writeLong(this.createdUnixDateTime);
@@ -406,6 +433,7 @@ public class FullDataSourceV2DTO
this.compressionModeValue = in.readByte();
this.applyToParent = in.readBoolean();
this.applyToChildren = in.readBoolean();
this.lastModifiedUnixDateTime = in.readLong();
this.createdUnixDateTime = in.readLong();
@@ -435,7 +463,7 @@ public class FullDataSourceV2DTO
{
return MoreObjects.toStringHelper(this)
.add("levelMinY", this.levelMinY)
.add("pos", this.pos)
.add("pos", DhSectionPos.toString(this.pos))
.add("dataChecksum", this.dataChecksum)
.add("compressedDataByteArray length", this.compressedDataByteArray.size())
.add("compressedColumnGenStepByteArray length", this.compressedColumnGenStepByteArray.size())
@@ -444,6 +472,7 @@ public class FullDataSourceV2DTO
.add("dataFormatVersion", this.dataFormatVersion)
.add("compressionModeValue", this.compressionModeValue)
.add("applyToParent", this.applyToParent)
.add("applyToChildren", this.applyToChildren)
.add("lastModifiedUnixDateTime", this.lastModifiedUnixDateTime)
.add("createdUnixDateTime", this.createdUnixDateTime)
.toString();
@@ -23,20 +23,18 @@ import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.sql.DatabaseUpdater;
import com.seibel.distanthorizons.core.sql.DbConnectionClosedException;
import com.seibel.distanthorizons.core.sql.dto.IBaseDTO;
import com.seibel.distanthorizons.core.sql.repo.phantoms.AutoClosableTrackingWrapper;
import com.seibel.distanthorizons.core.util.KeyedLockContainer;
import com.seibel.distanthorizons.core.util.ThreadUtil;
import com.seibel.distanthorizons.core.util.threading.ThreadPoolUtil;
import com.seibel.distanthorizons.coreapi.ModInfo;
import org.apache.logging.log4j.Logger;
import org.jetbrains.annotations.Nullable;
import javax.swing.plaf.nimbus.State;
import java.io.File;
import java.io.IOException;
import java.sql.*;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantLock;
/**
@@ -55,8 +53,6 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
private static final ConcurrentHashMap<String, Connection> CONNECTIONS_BY_CONNECTION_STRING = new ConcurrentHashMap<>();
private static final ConcurrentHashMap<AbstractDhRepo<?, ?>, String> ACTIVE_CONNECTION_STRINGS_BY_REPO = new ConcurrentHashMap<>();
private static final ThreadPoolExecutor WAL_FLUSH_THREAD = ThreadUtil.makeSingleDaemonThreadPool("Abstract Repo WAL Flush");
private static final AtomicBoolean FLUSH_THREAD_QUEUED = new AtomicBoolean(false);
private final String connectionString;
@@ -65,6 +61,8 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
public final String databaseType;
public final File databaseFile;
public final Set<AutoClosableTrackingWrapper> openClosables = ConcurrentHashMap.newKeySet();
public final Class<? extends TDTO> dtoClass;
protected final KeyedLockContainer<TKey> saveLockContainer = new KeyedLockContainer<>();
@@ -189,7 +187,15 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
}
catch (SQLException | IOException e)
{
LOGGER.warn("Unexpected issue deserializing DTO ["+this.dtoClass.getSimpleName()+"] with primary key ["+primaryKey+"]. Error: ["+e.getMessage()+"].", e);
if (e instanceof SQLException
&& DbConnectionClosedException.isClosedException((SQLException)e))
{
//LOGGER.warn("Attempted to get ["+this.dtoClass.getSimpleName()+"] with primary key ["+primaryKey+"] on closed repo ["+this.connectionString+"].");
}
else
{
LOGGER.warn("Unexpected issue deserializing DTO ["+this.dtoClass.getSimpleName()+"] with primary key ["+primaryKey+"]. Error: ["+e.getMessage()+"].", e);
}
return null;
}
}
@@ -310,65 +316,6 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
}
protected void tryTriggerWalFlush()
{
if (FLUSH_THREAD_QUEUED.compareAndSet(false, true))
{
WAL_FLUSH_THREAD.execute(() ->
{
try
{
Thread.sleep(10_000);
this.triggerWalFlush();
}
catch (InterruptedException ignore) { }
finally
{
FLUSH_THREAD_QUEUED.set(false);
}
});
}
}
protected void triggerWalFlush()
{
try (PreparedStatement statement = this.createPreparedStatement("PRAGMA wal_checkpoint(PASSIVE)");
ResultSet result = this.query(statement))
{
if (result == null)
{
return;
}
int busyInt = result.getInt("busy"); // usually 0 but will be 1 if a RESTART or FULL or TRUNCATE checkpoint was blocked from completing
boolean checkpointWasBlocked = (busyInt == 1);
int modifiedPageCount = result.getInt("log"); // number of modified pages that have been written to the write-ahead log file
int numberOfPagesWrittenToDb = result.getInt("checkpointed"); // number of pages in the write-ahead log file that have been successfully moved back into the database file at the conclusion of the checkpoint
if (!checkpointWasBlocked)
{
LOGGER.info("WAL flushed, modified pages: ["+modifiedPageCount+"], written pages: ["+numberOfPagesWrittenToDb+"].");
}
else
{
LOGGER.warn("WAL flush blocked, modified pages: ["+modifiedPageCount+"], written pages: ["+numberOfPagesWrittenToDb+"].");
}
}
catch (Exception e)
{
if (e instanceof SQLException && DbConnectionClosedException.IsClosedException((SQLException)e))
{
LOGGER.warn("DB closed");
}
else
{
LOGGER.error("unexpected error", e);
}
}
}
//==============//
// low level DB //
@@ -409,7 +356,7 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
// Note: this can only handle 1 command at a time
boolean resultSetPresent = statement.execute(sql);
try (ResultSet resultSet = statement.getResultSet())
try (ResultSet resultSet = AutoClosableTrackingWrapper.wrap(ResultSet.class, statement.getResultSet(), this.openClosables))
{
return this.convertResultSetToDictionaryList(resultSet, resultSetPresent);
}
@@ -419,7 +366,7 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
// SQL exceptions generally only happen when something is wrong with
// the database or the query and should cause the system to blow up to notify the developer
if (DbConnectionClosedException.IsClosedException(e))
if (DbConnectionClosedException.isClosedException(e))
{
throw new DbConnectionClosedException(e);
}
@@ -458,7 +405,8 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
boolean resultSetPresent = statement.execute();
if (resultSetPresent)
{
return statement.getResultSet();
ResultSet resultSet = statement.getResultSet();
return AutoClosableTrackingWrapper.wrap(ResultSet.class, resultSet, this.openClosables);
}
else
{
@@ -470,7 +418,7 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
// SQL exceptions generally only happen when something is wrong with
// the database or the query and should cause the system to blow up to notify the developer
if (DbConnectionClosedException.IsClosedException(e))
if (DbConnectionClosedException.isClosedException(e))
{
return null;
}
@@ -496,11 +444,11 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
{
PreparedStatement statement = this.connection.prepareStatement(sql);
statement.setQueryTimeout(TIMEOUT_SECONDS);
return statement;
return AutoClosableTrackingWrapper.wrap(PreparedStatement.class, statement, this.openClosables);
}
catch(SQLException e)
{
if (DbConnectionClosedException.IsClosedException(e))
if (DbConnectionClosedException.isClosedException(e))
{
return null;
}
@@ -554,7 +502,12 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
}
else
{
LOGGER.warn("Attempting to close already closed database connection: [" + connectionString + "]");
// these warnings can be ignored in release builds, as long as the connection is closed it doesn't really matter
// TODO fix duplicate closes
if (ModInfo.IS_DEV_BUILD)
{
LOGGER.warn("Attempting to close already closed database connection: [" + connectionString + "]");
}
}
}
}
@@ -581,6 +534,32 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
{
CONNECTIONS_BY_CONNECTION_STRING.remove(this.connectionString);
// log any leaked objects
int openClosableCount = this.openClosables.size();
if (openClosableCount != 0)
{
LOGGER.warn("[" + openClosableCount + "] objects not closed for repo [" + this.getClass().getSimpleName() + "]-[" + this.getTableName() + "] with connection: [" + this.connectionString + "]. A memory leak may be present and closing this connection may take longer than normal.");
// header
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append("Unclosed objects: \n");
// leaked objects
HashMap<String, AtomicInteger> unclosedObjectCountsByString = this.getUnclosedObjectStringsAndCounts();
for (String objString : unclosedObjectCountsByString.keySet())
{
AtomicInteger countRef = unclosedObjectCountsByString.get(objString);
if (countRef != null)
{
stringBuilder.append("[" + countRef.get() + "] - [" + objString + "] \n");
}
}
LOGGER.warn(stringBuilder.toString());
}
if (!this.connection.isClosed())
{
LOGGER.info("Closing database connection: [" + this.connectionString + "]...");
@@ -589,7 +568,12 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
}
else
{
LOGGER.warn("Attempting to close already closed database connection: [" + this.connectionString + "]");
// these warnings can be ignored in release builds, as long as the connection is closed it doesn't really matter
// TODO fix duplicate closes
if (ModInfo.IS_DEV_BUILD)
{
LOGGER.warn("Attempting to close already closed database connection: [" + this.connectionString + "]");
}
}
}
ACTIVE_CONNECTION_STRINGS_BY_REPO.remove(this);
@@ -675,6 +659,49 @@ public abstract class AbstractDhRepo<TKey, TDTO extends IBaseDTO<TKey>> implemen
return list;
}
/** used for logging leaked objects */
public HashMap<String, AtomicInteger> getUnclosedObjectStringsAndCounts()
{
HashMap<String, AtomicInteger> closableCountsByToString = new HashMap<>();
for (AutoClosableTrackingWrapper closableWrapper : this.openClosables)
{
// custom to-strings for better merging
String str = closableWrapper.wrappedClosable.getClass().getSimpleName();
if (closableWrapper.wrappedClosable instanceof ResultSet)
{
str += " @ " + closableWrapper.wrappedClosable.toString();
}
else if (closableWrapper.wrappedClosable instanceof PreparedStatement)
{
String sql = closableWrapper.wrappedClosable.toString();
int parametersIndex = sql.indexOf("\n parameters="); // remove Sqlite parameters so queries aren't separated by properties
if (parametersIndex != -1)
{
sql = sql.substring(0, parametersIndex);
}
str += " @ " + sql;
}
else
{
str += " @ " + closableWrapper.wrappedClosable.toString();
}
closableCountsByToString.compute(str, (stringVal, countRef) ->
{
if (countRef == null)
{
countRef = new AtomicInteger(0);
}
countRef.incrementAndGet();
return countRef;
});
}
return closableCountsByToString;
}
//==================//
@@ -174,8 +174,8 @@ public class BeaconBeamRepo extends AbstractDhRepo<DhBlockPos, BeaconBeamDTO>
int maxBlockZ = minBlockZ + LodUtil.CHUNK_WIDTH;
return this.getAllBeamsInBlockPosRange(
minBlockX, minBlockZ,
maxBlockX, maxBlockZ
minBlockX, maxBlockX,
minBlockZ, maxBlockZ
);
}
@@ -187,8 +187,8 @@ public class BeaconBeamRepo extends AbstractDhRepo<DhBlockPos, BeaconBeamDTO>
int maxBlockZ = minBlockZ + DhSectionPos.getBlockWidth(pos);
return this.getAllBeamsInBlockPosRange(
minBlockX, minBlockZ,
maxBlockX, maxBlockZ
minBlockX, maxBlockX,
minBlockZ, maxBlockZ
);
}
@@ -199,8 +199,8 @@ public class BeaconBeamRepo extends AbstractDhRepo<DhBlockPos, BeaconBeamDTO>
"? <= BlockPosX AND BlockPosX <= ? AND " +
"? <= BlockPosZ AND BlockPosZ <= ?";
public List<BeaconBeamDTO> getAllBeamsInBlockPosRange(
int minBlockX, int minBlockZ,
int maxBlockX, int maxBlockZ
int minBlockX, int maxBlockX,
int minBlockZ, int maxBlockZ
)
{
ArrayList<BeaconBeamDTO> beamList = new ArrayList<>();
@@ -214,8 +214,8 @@ public class BeaconBeamRepo extends AbstractDhRepo<DhBlockPos, BeaconBeamDTO>
int i = 1;
statement.setInt(i++, minBlockX);
statement.setInt(i++, minBlockZ);
statement.setInt(i++, maxBlockX);
statement.setInt(i++, minBlockZ);
statement.setInt(i++, maxBlockZ);
@@ -25,12 +25,12 @@ import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.core.pos.DhSectionPos;
import com.seibel.distanthorizons.core.sql.DbConnectionClosedException;
import com.seibel.distanthorizons.core.sql.dto.FullDataSourceV2DTO;
import com.seibel.distanthorizons.core.util.BoolUtil;
import com.seibel.distanthorizons.core.util.ListUtil;
import com.seibel.distanthorizons.core.util.objects.dataStreams.DhDataInputStream;
import it.unimi.dsi.fastutil.bytes.ByteArrayList;
import it.unimi.dsi.fastutil.longs.LongArrayList;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.util.ArrayUtils;
import org.jetbrains.annotations.Nullable;
import java.io.*;
@@ -81,7 +81,7 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
@Nullable
@Override @Nullable
public FullDataSourceV2DTO convertResultSetToDto(ResultSet resultSet) throws ClassCastException, IOException, SQLException
{
//======================//
@@ -101,7 +101,9 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
byte dataFormatVersion = resultSet.getByte("DataFormatVersion");
byte compressionModeValue = resultSet.getByte("CompressionMode");
// while these values can be null in the DB, null would just equate to false
boolean applyToParent = (resultSet.getInt("ApplyToParent")) == 1;
boolean applyToChildren = (resultSet.getInt("ApplyToChildren")) == 1;
long lastModifiedUnixDateTime = resultSet.getLong("LastModifiedUnixDateTime");
long createdUnixDateTime = resultSet.getLong("CreatedUnixDateTime");
@@ -129,6 +131,7 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
dto.lastModifiedUnixDateTime = lastModifiedUnixDateTime;
dto.createdUnixDateTime = createdUnixDateTime;
dto.applyToParent = applyToParent;
dto.applyToChildren = applyToChildren;
dto.levelMinY = minY;
}
return dto;
@@ -139,13 +142,13 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
" DetailLevel, PosX, PosZ, \n" +
" MinY, DataChecksum, \n" +
" Data, ColumnGenerationStep, ColumnWorldCompressionMode, Mapping, \n" +
" DataFormatVersion, CompressionMode, ApplyToParent, \n" +
" DataFormatVersion, CompressionMode, ApplyToParent, ApplyToChildren, \n" +
" LastModifiedUnixDateTime, CreatedUnixDateTime) \n" +
"VALUES( \n" +
" ?, ?, ?, \n" +
" ?, ?, \n" +
" ?, ?, ?, ?, \n" +
" ?, ?, ?, \n" +
" ?, ?, ?, ?, \n" +
" ?, ? \n" +
");";
@Override
@@ -159,51 +162,63 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
int i = 1;
statement.setObject(i++, DhSectionPos.getDetailLevel(dto.pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL);
statement.setObject(i++, DhSectionPos.getX(dto.pos));
statement.setObject(i++, DhSectionPos.getZ(dto.pos));
statement.setInt(i++, DhSectionPos.getDetailLevel(dto.pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL);
statement.setInt(i++, DhSectionPos.getX(dto.pos));
statement.setInt(i++, DhSectionPos.getZ(dto.pos));
statement.setObject(i++, dto.levelMinY);
statement.setObject(i++, dto.dataChecksum);
statement.setInt(i++, dto.levelMinY);
statement.setInt(i++, dto.dataChecksum);
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedDataByteArray.elements()), dto.compressedDataByteArray.size());
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedColumnGenStepByteArray.elements()), dto.compressedColumnGenStepByteArray.size());
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedWorldCompressionModeByteArray.elements()), dto.compressedWorldCompressionModeByteArray.size());
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedMappingByteArray.elements()), dto.compressedMappingByteArray.size());
statement.setObject(i++, dto.dataFormatVersion);
statement.setObject(i++, dto.compressionModeValue);
statement.setObject(i++, dto.applyToParent);
statement.setByte(i++, dto.dataFormatVersion);
statement.setByte(i++, dto.compressionModeValue);
// if nothing is present assume we don't need/want to propagate updates
statement.setBoolean(i++, BoolUtil.falseIfNull(dto.applyToParent));
statement.setBoolean(i++, BoolUtil.falseIfNull(dto.applyToChildren));
statement.setObject(i++, System.currentTimeMillis()); // last modified unix time
statement.setObject(i++, System.currentTimeMillis()); // created unix time
statement.setLong(i++, System.currentTimeMillis()); // last modified unix time
statement.setLong(i++, System.currentTimeMillis()); // created unix time
return statement;
}
private final String updateSqlTemplate =
"UPDATE "+this.getTableName()+" \n" +
"SET \n" +
" MinY = ? \n" +
" ,DataChecksum = ? \n" +
" ,Data = ? \n" +
" ,ColumnGenerationStep = ? \n" +
" ,ColumnWorldCompressionMode = ? \n" +
" ,Mapping = ? \n" +
" ,DataFormatVersion = ? \n" +
" ,CompressionMode = ? \n" +
" ,ApplyToParent = ? \n" +
" ,LastModifiedUnixDateTime = ? \n" +
" ,CreatedUnixDateTime = ? \n" +
"WHERE DetailLevel = ? AND PosX = ? AND PosZ = ?";
@Override
public PreparedStatement createUpdateStatement(FullDataSourceV2DTO dto) throws SQLException
{
PreparedStatement statement = this.createPreparedStatement(this.updateSqlTemplate);
// Dynamic string so we can update one, both, or neither
// of the applyTo... flags.
// This is necessary to prevent concurrent modifications when
// update propagation is run.
String updateSqlTemplate = (
"UPDATE "+this.getTableName()+" \n" +
"SET \n" +
" MinY = ? \n" +
" ,DataChecksum = ? \n" +
" ,Data = ? \n" +
" ,ColumnGenerationStep = ? \n" +
" ,ColumnWorldCompressionMode = ? \n" +
" ,Mapping = ? \n" +
" ,DataFormatVersion = ? \n" +
" ,CompressionMode = ? \n" +
// only update these values if they're present
(dto.applyToParent != null ? " ,ApplyToParent = ? \n" : "" ) +
(dto.applyToChildren != null ? " ,ApplyToChildren = ? \n" : "" ) +
" ,LastModifiedUnixDateTime = ? \n" +
" ,CreatedUnixDateTime = ? \n" +
"WHERE DetailLevel = ? AND PosX = ? AND PosZ = ?"
// intern should help reduce memory overhead due to this string being dynamic
).intern();
PreparedStatement statement = this.createPreparedStatement(updateSqlTemplate);
if (statement == null)
{
return null;
@@ -211,24 +226,31 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
int i = 1;
statement.setObject(i++, dto.levelMinY);
statement.setObject(i++, dto.dataChecksum);
statement.setInt(i++, dto.levelMinY);
statement.setInt(i++, dto.dataChecksum);
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedDataByteArray.elements()), dto.compressedDataByteArray.size());
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedColumnGenStepByteArray.elements()), dto.compressedColumnGenStepByteArray.size());
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedWorldCompressionModeByteArray.elements()), dto.compressedWorldCompressionModeByteArray.size());
statement.setBinaryStream(i++, new ByteArrayInputStream(dto.compressedMappingByteArray.elements()), dto.compressedMappingByteArray.size());
statement.setObject(i++, dto.dataFormatVersion);
statement.setObject(i++, dto.compressionModeValue);
statement.setObject(i++, dto.applyToParent);
statement.setByte(i++, dto.dataFormatVersion);
statement.setByte(i++, dto.compressionModeValue);
if (dto.applyToParent != null)
{
statement.setBoolean(i++, dto.applyToParent);
}
if (dto.applyToChildren != null)
{
statement.setBoolean(i++, dto.applyToChildren);
}
statement.setObject(i++, System.currentTimeMillis()); // last modified unix time
statement.setObject(i++, dto.createdUnixDateTime);
statement.setLong(i++, System.currentTimeMillis()); // last modified unix time
statement.setLong(i++, dto.createdUnixDateTime);
statement.setObject(i++, DhSectionPos.getDetailLevel(dto.pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL);
statement.setObject(i++, DhSectionPos.getX(dto.pos));
statement.setObject(i++, DhSectionPos.getZ(dto.pos));
statement.setInt(i++, DhSectionPos.getDetailLevel(dto.pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL);
statement.setInt(i++, DhSectionPos.getX(dto.pos));
statement.setInt(i++, DhSectionPos.getZ(dto.pos));
return statement;
}
@@ -239,23 +261,35 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
// updates //
//=========//
/** should be be very similar to {@link FullDataSourceV2Repo#setApplyToChildrenSql} */
private final String setApplyToParentSql =
"UPDATE "+this.getTableName()+" \n" +
"SET ApplyToParent = ? \n" +
"WHERE DetailLevel = ? AND PosX = ? AND PosZ = ?";
public void setApplyToParent(long pos, boolean applyToParent)
{ this.setApplyToFlag(pos, applyToParent, true); }
/** should be be very similar to {@link FullDataSourceV2Repo#setApplyToParentSql} */
private final String setApplyToChildrenSql =
"UPDATE "+this.getTableName()+" \n" +
"SET ApplyToChildren = ? \n" +
"WHERE DetailLevel = ? AND PosX = ? AND PosZ = ?";
public void setApplyToChild(long pos, boolean applyToChild)
{ this.setApplyToFlag(pos, applyToChild, false); }
private void setApplyToFlag(long pos, boolean applyFlag, boolean applyToParent)
{
PreparedStatement statement = this.createPreparedStatement(this.setApplyToParentSql);
if (statement == null)
{
return;
}
try
String sql = applyToParent ? this.setApplyToParentSql : this.setApplyToChildrenSql;
try (PreparedStatement statement = this.createPreparedStatement(sql))
{
if (statement == null)
{
return;
}
int i = 1;
statement.setBoolean(i++, applyToParent);
statement.setBoolean(i++, applyFlag);
int detailLevel = DhSectionPos.getDetailLevel(pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL;
statement.setInt(i++, detailLevel);
@@ -273,26 +307,43 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
}
}
private final String getPositionsToUpdateSql =
/** should be be very similar to {@link FullDataSourceV2Repo#getChildPositionsToUpdateSql} */
private final String getParentPositionsToUpdateSql =
"SELECT DetailLevel, PosX, PosZ, " +
" (sqrt(pow(PosX - ?, 2) + pow(PosZ - ?, 2))) AS Distance " +
"FROM "+this.getTableName()+" " +
"WHERE ApplyToParent = 1 " +
"ORDER BY Distance ASC " +
"LIMIT ?; ";
" abs((PosX << (6 + DetailLevel)) - ?) + abs((PosZ << (6 + DetailLevel)) - ?) AS Distance " +
"FROM " + this.getTableName() + " " +
"WHERE ApplyToParent = 1 " +
"ORDER BY DetailLevel ASC, Distance ASC " +
"LIMIT ?; ";
public LongArrayList getPositionsToUpdate(int targetBlockPosX, int targetBlockPosZ, int returnCount)
{ return this.getPositionsToUpdate(targetBlockPosX, targetBlockPosZ, returnCount, true); }
/** should be be very similar to {@link FullDataSourceV2Repo#getParentPositionsToUpdateSql} */
private final String getChildPositionsToUpdateSql =
"SELECT DetailLevel, PosX, PosZ, " +
" abs((PosX << (6 + DetailLevel)) - ?) + abs((PosZ << (6 + DetailLevel)) - ?) AS Distance " +
"FROM " + this.getTableName() + " " +
"WHERE ApplyToChildren = 1 " +
"ORDER BY DetailLevel ASC, Distance ASC " +
"LIMIT ?; ";
public LongArrayList getChildPositionsToUpdate(int targetBlockPosX, int targetBlockPosZ, int returnCount)
{ return this.getPositionsToUpdate(targetBlockPosX, targetBlockPosZ, returnCount, false); }
private LongArrayList getPositionsToUpdate(int targetBlockPosX, int targetBlockPosZ, int returnCount, boolean getParentUpdates)
{
LongArrayList list = new LongArrayList();
PreparedStatement statement = this.createPreparedStatement(this.getPositionsToUpdateSql);
if (statement == null)
{
return list;
}
try
String sql = getParentUpdates ? this.getParentPositionsToUpdateSql : this.getChildPositionsToUpdateSql;
try (PreparedStatement statement = this.createPreparedStatement(sql))
{
if (statement == null)
{
return list;
}
int i = 1;
statement.setInt(i++, targetBlockPosX);
statement.setInt(i++, targetBlockPosZ);
@@ -322,6 +373,7 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
}
private final String getColumnGenerationStepSql =
"select ColumnGenerationStep, CompressionMode " +
"from "+this.getTableName()+" " +
@@ -329,15 +381,14 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
/** @return null if nothing exists for this position */
public void getColumnGenerationStepForPos(long pos, ByteArrayList outputByteArray)
{
PreparedStatement statement = this.createPreparedStatement(this.getColumnGenerationStepSql);
if (statement == null)
{
return;
}
try
try (PreparedStatement statement = this.createPreparedStatement(this.getColumnGenerationStepSql))
{
if (statement == null)
{
return;
}
int detailLevel = DhSectionPos.getDetailLevel(pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL;
@@ -395,9 +446,8 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
@Nullable
public Long getTimestampForPos(long pos)
{
try
try(PreparedStatement preparedStatement = this.createPreparedStatement(this.getTimestampForPosSql))
{
PreparedStatement preparedStatement = this.createPreparedStatement(this.getTimestampForPosSql);
if (preparedStatement == null)
{
return null;
@@ -437,9 +487,8 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
"AND PosZ BETWEEN ? AND ?;";
public Map<Long, Long> getTimestampsForRange(byte detailLevel, int startPosX, int startPosZ, int endPosX, int endPosZ)
{
try
try(PreparedStatement preparedStatement = this.createPreparedStatement(this.getTimestampForRangeSql))
{
PreparedStatement preparedStatement = this.createPreparedStatement(this.getTimestampForRangeSql);
if (preparedStatement == null)
{
return new HashMap<>();
@@ -487,28 +536,30 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
public LongArrayList getAllPositions()
{
LongArrayList list = new LongArrayList();
PreparedStatement statement = this.createPreparedStatement(getAllPositionsSql);
if (statement == null)
{
return list;
}
try(ResultSet result = this.query(statement))
try (PreparedStatement statement = this.createPreparedStatement(this.getAllPositionsSql))
{
while (result != null && result.next())
if (statement == null)
{
byte detailLevel = result.getByte("DetailLevel");
byte sectionDetailLevel = (byte) (detailLevel + DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL);
int posX = result.getInt("PosX");
int posZ = result.getInt("PosZ");
long pos = DhSectionPos.encode(sectionDetailLevel, posX, posZ);
list.add(pos);
return list;
}
return list;
try(ResultSet result = this.query(statement))
{
while (result != null && result.next())
{
byte detailLevel = result.getByte("DetailLevel");
byte sectionDetailLevel = (byte) (detailLevel + DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL);
int posX = result.getInt("PosX");
int posZ = result.getInt("PosZ");
long pos = DhSectionPos.encode(sectionDetailLevel, posX, posZ);
list.add(pos);
}
return list;
}
}
catch (SQLException e)
{
@@ -529,14 +580,13 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
{
int detailLevel = DhSectionPos.getDetailLevel(pos) - DhSectionPos.SECTION_MINIMUM_DETAIL_LEVEL;
PreparedStatement statement = this.createPreparedStatement(this.getDataSizeInBytesSql);
if (statement == null)
try (PreparedStatement statement = this.createPreparedStatement(this.getDataSizeInBytesSql))
{
return 0L;
}
if (statement == null)
{
return 0L;
}
try
{
int i = 1;
statement.setInt(i++, detailLevel);
statement.setInt(i++, DhSectionPos.getX(pos));
@@ -565,9 +615,8 @@ public class FullDataSourceV2Repo extends AbstractDhRepo<Long, FullDataSourceV2D
/** @return the total size in bytes of the full data for this entire database */
public long getTotalDataSizeInBytes()
{
PreparedStatement statement = this.createPreparedStatement(this.getTotalDataSizeInBytesSql);
try(ResultSet result = this.query(statement))
try (PreparedStatement statement = this.createPreparedStatement(this.getTotalDataSizeInBytesSql);
ResultSet result = this.query(statement))
{
if (result == null || !result.next())
{
@@ -0,0 +1,102 @@
package com.seibel.distanthorizons.core.sql.repo.phantoms;
import com.seibel.distanthorizons.coreapi.ModInfo;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.Set;
/**
* This is used to detect leaks
* with our JDBC implementation, specifically
* to make sure all {@link AutoCloseable} objects
* are inside try-finally resources.
*/
public class AutoClosableTrackingWrapper implements InvocationHandler
{
//private static final Logger LOGGER = DhLoggerBuilder.getLogger();
/**
* should be enabled during development to
* notify if any resources are being leaked.
*/
public static final boolean TRACK_WRAPPERS = ModInfo.IS_DEV_BUILD;
@NotNull
public final AutoCloseable wrappedClosable;
@NotNull
public final Set<AutoClosableTrackingWrapper> parentTrackingSet;
//==============//
// constructors //
//==============//
@Nullable
public static <TStatic extends AutoCloseable> TStatic wrap(
@NotNull Class<?> clazz,
@Nullable TStatic wrappedClosable,
@NotNull Set<AutoClosableTrackingWrapper> trackingSet)
{
if (!TRACK_WRAPPERS)
{
return wrappedClosable;
}
// done to prevent null pointers
if (wrappedClosable == null)
{
return null;
}
return (TStatic) Proxy.newProxyInstance(
wrappedClosable.getClass().getClassLoader(),
new Class[]{ clazz },
new AutoClosableTrackingWrapper(wrappedClosable, trackingSet)
);
}
private AutoClosableTrackingWrapper(@NotNull AutoCloseable wrappedClosable, @NotNull Set<AutoClosableTrackingWrapper> trackingSet)
{
this.wrappedClosable = wrappedClosable;
this.parentTrackingSet = trackingSet;
this.parentTrackingSet.add(this);
}
//============//
// reflection //
//============//
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
{
// Track the close() method
if ("close".equals(method.getName()))
{
this.wrappedClosable.close();
this.parentTrackingSet.remove(this);
return null;
}
try
{
// Delegate all other methods to the wrapped object
return method.invoke(this.wrappedClosable, args);
}
catch (InvocationTargetException e)
{
// get the target so we can filter the exception correctly up-stream
throw e.getTargetException();
}
}
}
@@ -0,0 +1,20 @@
package com.seibel.distanthorizons.core.util;
public class BoolUtil
{
/** Used to prevent null {@link Boolean} objects in if statements */
public static boolean falseIfNull(Boolean value)
{
if (value == null)
{
// default to false since null doesn't mean true in any context
// (Even in JavaScript)
return false;
}
else
{
return value;
}
}
}
@@ -0,0 +1,16 @@
package com.seibel.distanthorizons.core.util;
import java.time.Duration;
public class FormatUtil
{
public static String formatEta(Duration duration)
{
return duration.toString()
.substring(2)
.replaceAll("(\\d[HMS])(?!$)", "$1 ")
.replaceAll("\\.\\d+", "")
.toLowerCase();
}
}
@@ -137,8 +137,6 @@ public class RenderDataPointReducingList extends PhantomArrayListParent
this.sortingArray = this.pooledArraysCheckout.getShortArray(0, 0);
if (ASSERTS) this.checkLinks();
this.pooledArraysCheckout = null;
return;
}
@@ -154,8 +152,6 @@ public class RenderDataPointReducingList extends PhantomArrayListParent
java.util.Arrays.fill(this.links.elements(), DEFAULT_LINKS);
this.data = this.pooledArraysCheckout.getLongArray(1, arrayCapacity);
this.pooledArraysCheckout = null;
int sizeWithoutAir = 0;
for (int index = 0; index < size; index++)
{
@@ -73,7 +73,10 @@ public class RenderDataPointUtil
public final static int EMPTY_DATA = 0;
public final static int MAX_WORLD_Y_SIZE = 4096;
// the maximum valid Y value is the maximum min y + world height.
// min y is [-2032, 2031], height is < 4064.
public final static int MAX_WORLD_Y_SIZE = 2031 + 4064;
public final static int ALPHA_DOWNSIZE_SHIFT = 4;
@@ -92,7 +92,9 @@ public class RenderUtil
// but far enough the fading will rarely overlap (IE only at extreme FOV)
return getNearClipPlaneDistanceInBlocks(partialTicks, 0.2f);
}
public static float getNearClipPlaneInBlocksForFading(float partialTicks)
/** TODO this should be moved into the config file or something, this is confusing and obtuse to use */
@Deprecated
public static float getAutoOverdrawPrevention()
{
float overdraw = Config.Client.Advanced.Graphics.Culling.overdrawPrevention.get().floatValue();
@@ -123,6 +125,11 @@ public class RenderUtil
}
}
return overdraw;
}
public static float getNearClipPlaneInBlocksForFading(float partialTicks)
{
float overdraw = getAutoOverdrawPrevention();
return getNearClipPlaneDistanceInBlocks(partialTicks, overdraw);
}
private static float getNearClipPlaneDistanceInBlocks(float partialTicks, float overdrawPreventionPercent)
@@ -223,6 +230,9 @@ public class RenderUtil
return "No Client World Loaded";
}
// TODO changing to getOrLoadClientLevel() fixes Immersive Portals only rendering the level the user starts in
// however this may break how other level handling is done so James doesn't want to change it.
// Special handling may be necessary when Immersive Portals is present, although additional testing is needed.
IDhClientLevel level = clientWorld.getClientLevel(levelWrapper);
if (level == null)
{
@@ -1,430 +0,0 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.core.util.objects;
import com.seibel.distanthorizons.core.logging.DhLoggerBuilder;
import com.seibel.distanthorizons.coreapi.ModInfo;
import org.apache.logging.log4j.Logger;
import java.lang.invoke.MethodHandles;
import java.util.HashMap;
import java.util.function.Function;
/**
* Handles parsing and creating string messages from OpenGL messages.
*
* @author Leetom
* @version 2022-10-1
*/
public final class GLMessage
{
private static final Logger LOGGER = DhLoggerBuilder.getLogger(MethodHandles.lookup().lookupClass().getSimpleName());
static final String HEADER = "[LWJGL] OpenGL debug message";
public final EType type;
public final ESeverity severity;
public final ESource source;
public final String id;
public final String message;
/** This is needed since gl callback will not have the correct class loader set, which causes issues. */
static void initLoadClass()
{
Builder dummy = new Builder();
dummy.add(GLMessage.HEADER);
dummy.add("ID");
dummy.add(":");
dummy.add("dummyId");
dummy.add("Source");
dummy.add(":");
dummy.add(ESource.API.name);
dummy.add("Type");
dummy.add(":");
dummy.add(EType.OTHER.name);
dummy.add("Severity");
dummy.add(":");
dummy.add(ESeverity.LOW.name);
dummy.add("Message");
dummy.add(":");
dummy.add("dummyMessage");
}
static
{
initLoadClass();
}
GLMessage(EType type, ESeverity severity, ESource source, String id, String message)
{
this.type = type;
this.source = source;
this.severity = severity;
this.id = id;
this.message = message;
}
@Override
public String toString() { return "[level:" + severity + ", type:" + type + ", source:" + source + ", id:" + id + ", msg:{" + message + "}]"; }
//==============//
// helper enums //
//==============//
public enum EType
{
ERROR,
DEPRECATED_BEHAVIOR,
UNDEFINED_BEHAVIOR,
PORTABILITY,
PERFORMANCE,
MARKER,
PUSH_GROUP,
POP_GROUP,
OTHER;
private static final HashMap<String, EType> ENUM_BY_NAME = new HashMap<>();
private final String name;
static
{
for (EType type : EType.values())
{
ENUM_BY_NAME.put(type.name, type);
}
}
EType() { name = super.toString().toUpperCase(); }
@Override
public final String toString() { return name; }
public static EType get(String name) { return ENUM_BY_NAME.get(name.toUpperCase()); }
}
public enum ESource
{
API,
WINDOW_SYSTEM,
SHADER_COMPILER,
THIRD_PARTY,
APPLICATION,
OTHER;
private static final HashMap<String, ESource> ENUM_BY_NAME = new HashMap<>();
public final String name;
static
{
for (ESource source : ESource.values())
{
ENUM_BY_NAME.put(source.name, source);
}
}
ESource() { name = super.toString().toUpperCase(); }
@Override
public final String toString() { return name; }
public static ESource get(String name) { return ENUM_BY_NAME.get(name.toUpperCase()); }
}
public enum ESeverity
{
HIGH,
MEDIUM,
LOW,
NOTIFICATION;
public final String name;
static final HashMap<String, ESeverity> ENUM_BY_NAME = new HashMap<>();
static
{
for (ESeverity severity : ESeverity.values())
{
ENUM_BY_NAME.put(severity.name, severity);
}
}
ESeverity() { name = super.toString().toUpperCase(); }
@Override
public final String toString() { return name; }
public static ESeverity get(String name) { return ENUM_BY_NAME.get(name.toUpperCase()); }
}
//================//
// helper classes //
//================//
/**
* Expected message format: <br>
* <code>
* [LWJGL] OpenGL debug message <br>
* ID: 0x20071 <br>
* Source: API <br>
* Type: OTHER <br>
* Severity: NOTIFICATION <br>
* Message: Buffer detailed info: Buffer object 1014084 (bound to ...
* </code>
*/
public static class Builder
{
/** how many stages are present in the message parser */
private static final int FINAL_PARSER_STAGE_INDEX = 15;
public static final Builder DEFAULT_MESSAGE_BUILDER =
new Builder(
(type) ->
{ // type filter
if (type == GLMessage.EType.POP_GROUP)
return false;
if (type == GLMessage.EType.PUSH_GROUP)
return false;
if (type == GLMessage.EType.MARKER)
return false;
// if (type == GLMessage.Type.PERFORMANCE) return false;
return true;
},
(severity) ->
{ // severity filter
if (severity == GLMessage.ESeverity.NOTIFICATION)
return false;
return true;
},
null
);
private final StringBuilder inProgressMessageBuilder = new StringBuilder();
private EType type;
private ESeverity severity;
private ESource source;
/** if the function returns false the message will be allowed */
private final Function<EType, Boolean> typeFilter;
/** if the function returns false the message will be allowed */
private final Function<ESeverity, Boolean> severityFilter;
/** if the function returns false the message will be allowed */
private final Function<ESource, Boolean> sourceFilter;
private String id;
private String message;
/** how far into the message parser this builder is */
private int parserStage = 0;
static
{
initLoadClass();
}
public Builder() { this(null, null, null); }
public Builder(
Function<EType, Boolean> typeFilter,
Function<ESeverity, Boolean> severityFilter,
Function<ESource, Boolean> sourceFilter)
{
this.typeFilter = typeFilter;
this.severityFilter = severityFilter;
this.sourceFilter = sourceFilter;
}
/**
* Adds the given string to the message builder. <br> <br>
*
* Will log a warning if the string given wasn't expected
* for the next stage of the OpenGL message format.<br> <br>
*
* @return null if the message isn't complete
*/
public GLMessage add(String str)
{
// TODO fix implementation for MC 1.20.2 and newer
// please see the incomplete GLMessageTest for an example as to how the message formats differ
if (true)
return null;
str = str.trim();
if (str.isEmpty())
return null;
boolean parseSuccess = runNextParserStage(str);
if (parseSuccess && parserStage > FINAL_PARSER_STAGE_INDEX)
{
this.parserStage = 0;
GLMessage msg = new GLMessage(this.type, this.severity, this.source, this.id, this.message);
if (doesMessagePassFilters(msg))
{
return msg;
}
}
else if (!parseSuccess)
{
LOGGER.warn("Failed to parse GLMessage line '{}' at stage {}", str, parserStage);
}
// the message isn't finished yet
return null;
// TODO implement a method that works for both MC 1.20.2+ and 1.20.1-
//if (str.equals(HEADER) && inProgressMessageBuilder.length() != 0)
//{
// boolean parseSuccess = runNextParserStage(str);
// if (parseSuccess && parserStage > FINAL_PARSER_STAGE_INDEX)
// {
// this.parserStage = 0;
// GLMessage msg = new GLMessage(this.type, this.severity, this.source, this.id, this.message);
// if (doesMessagePassFilters(msg))
// {
// return msg;
// }
// else
// {
// inProgressMessageBuilder.setLength(0);
// return null;
// }
// }
// else
// {
// if (!parseSuccess)
// {
// LOGGER.warn("Failed to parse GLMessage line '{}' at stage {}", str, parserStage);
// inProgressMessageBuilder.setLength(0);
// }
//
// return null;
// }
//}
//else
//{
// inProgressMessageBuilder.append(str);
// return null;
//}
}
private boolean doesMessagePassFilters(GLMessage msg)
{
if (this.sourceFilter != null && !this.sourceFilter.apply(msg.source))
return false;
else if (this.typeFilter != null && !this.typeFilter.apply(msg.type))
return false;
else if (this.severityFilter != null && !this.severityFilter.apply(msg.severity))
return false;
else
return true;
}
/** @return true if the given string was expected next for the OpenGL message format */
private boolean runNextParserStage(String str)
{
switch (this.parserStage)
{
case 0:
return checkAndIncStage(str, GLMessage.HEADER);
case 1:
return checkAndIncStage(str, "ID");
case 2:
return checkAndIncStage(str, ":");
case 3:
this.id = str;
this.parserStage++;
return true;
case 4:
return checkAndIncStage(str, "Source");
case 5:
return checkAndIncStage(str, ":");
case 6:
this.source = ESource.get(str);
this.parserStage++;
return true;
case 7:
return checkAndIncStage(str, "Type");
case 8:
return checkAndIncStage(str, ":");
case 9:
this.type = EType.get(str);
this.parserStage++;
return true;
case 10:
return checkAndIncStage(str, "Severity");
case 11:
return checkAndIncStage(str, ":");
case 12:
this.severity = ESeverity.get(str);
this.parserStage++;
return true;
case 13:
return checkAndIncStage(str, "Message");
case 14:
return checkAndIncStage(str, ":");
case 15:
this.message = str;
this.parserStage++;
return true;
default:
return false;
}
}
/**
* Returns true and increments the parserStage
* if the given and expected strings are the same.
*/
private boolean checkAndIncStage(String givenString, String expectedString)
{
boolean equal = givenString.equals(expectedString);
//boolean equal = givenString.contains(expectedString);
if (equal)
this.parserStage++;
return equal;
}
} // builder class
}
@@ -0,0 +1,54 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.core.util.objects.GLMessages;
import java.util.HashMap;
public enum EGLMessageSeverity
{
HIGH,
MEDIUM,
LOW,
NOTIFICATION;
public final String name;
static final HashMap<String, EGLMessageSeverity> ENUM_BY_NAME = new HashMap<>();
static
{
for (EGLMessageSeverity severity : EGLMessageSeverity.values())
{
ENUM_BY_NAME.put(severity.name, severity);
}
}
EGLMessageSeverity() { this.name = super.toString().toUpperCase(); }
@Override
public final String toString() { return this.name; }
public static EGLMessageSeverity get(String name) { return ENUM_BY_NAME.get(name.toUpperCase()); }
}
@@ -0,0 +1,55 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.core.util.objects.GLMessages;
import java.util.HashMap;
public enum EGLMessageSource
{
API,
WINDOW_SYSTEM,
SHADER_COMPILER,
THIRD_PARTY,
APPLICATION,
OTHER;
private static final HashMap<String, EGLMessageSource> ENUM_BY_NAME = new HashMap<>();
public final String name;
static
{
for (EGLMessageSource source : EGLMessageSource.values())
{
ENUM_BY_NAME.put(source.name, source);
}
}
EGLMessageSource() { this.name = super.toString().toUpperCase(); }
@Override
public final String toString() { return this.name; }
public static EGLMessageSource get(String name) { return ENUM_BY_NAME.get(name.toUpperCase()); }
}
@@ -0,0 +1,58 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.core.util.objects.GLMessages;
import java.util.HashMap;
public enum EGLMessageType
{
ERROR,
DEPRECATED_BEHAVIOR,
UNDEFINED_BEHAVIOR,
PORTABILITY,
PERFORMANCE,
MARKER,
PUSH_GROUP,
POP_GROUP,
OTHER;
private static final HashMap<String, EGLMessageType> ENUM_BY_NAME = new HashMap<>();
public final String name;
static
{
for (EGLMessageType type : EGLMessageType.values())
{
ENUM_BY_NAME.put(type.name, type);
}
}
EGLMessageType() { this.name = super.toString().toUpperCase(); }
@Override
public final String toString() { return this.name; }
public static EGLMessageType get(String name) { return ENUM_BY_NAME.get(name.toUpperCase()); }
}
@@ -0,0 +1,56 @@
/*
* This file is part of the Distant Horizons mod
* licensed under the GNU LGPL v3 License.
*
* Copyright (C) 2020 James Seibel
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, version 3.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package com.seibel.distanthorizons.core.util.objects.GLMessages;
public final class GLMessage
{
static final String HEADER = "[LWJGL] OpenGL debug message";
public final EGLMessageType type;
public final EGLMessageSeverity severity;
public final EGLMessageSource source;
public final String id;
public final String message;
GLMessage(EGLMessageType type, EGLMessageSeverity severity, EGLMessageSource source, String id, String message)
{
this.type = type;
this.source = source;
this.severity = severity;
this.id = id;
this.message = message;
}
@Override
public String toString()
{
return "level: [" + this.severity + "], " +
"type: [" + this.type + "], " +
"source: [" + this.source + "], " +
"id: [" + this.id + "], " +
"msg: [" + this.message + "]";
}
}

Some files were not shown because too many files have changed in this diff Show More