Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[APV] Cell streaming system #5731

Merged
merged 108 commits into from
Oct 20, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
75c954a
Updated MeshGizmo to handle both traingles and lines.
JulienIgnace-Unity Aug 5, 2021
570493f
Fixed cell and brick debug scale
JulienIgnace-Unity Aug 5, 2021
1bfc541
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Aug 11, 2021
53f4f3f
With a null ref exception when applying virtual offset on unloaded sc…
JulienIgnace-Unity Aug 13, 2021
70a3d3d
Fixed an issue causing failure of the bake when multiple scenes are l…
JulienIgnace-Unity Aug 13, 2021
d381f78
Faster new version of debug probes with compute buffers instead of co…
JulienIgnace-Unity Aug 17, 2021
00a460e
Fixed debug probes after domain relaod when using L1
JulienIgnace-Unity Aug 17, 2021
e128c24
Fixed compute buffer probe debug
JulienIgnace-Unity Aug 17, 2021
51eac63
Matched cell debug culling with cell culling used for probe debug.
JulienIgnace-Unity Aug 17, 2021
c3160d6
Revert "Matched cell debug culling with cell culling used for probe d…
JulienIgnace-Unity Aug 17, 2021
31180f1
Fixed per probe culling.
JulienIgnace-Unity Aug 17, 2021
0a7730f
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Aug 18, 2021
78596c7
Revert "Fixed compute buffer probe debug"
JulienIgnace-Unity Aug 18, 2021
72a75b4
Revert "Faster new version of debug probes with compute buffers inste…
JulienIgnace-Unity Aug 18, 2021
1cbd0b5
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Aug 19, 2021
b44cde7
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Aug 19, 2021
f762552
Merge branch 'hd/apv-msic-fixes' of https://github.com/Unity-Technolo…
JulienIgnace-Unity Aug 19, 2021
4c7f031
Cleaning in preparation for streaming
JulienIgnace-Unity Aug 19, 2021
f0d5e5c
More cleanup + Fix debug upon load/unload of cells
JulienIgnace-Unity Aug 19, 2021
4fdb748
Fix GCAlloc
JulienIgnace-Unity Aug 23, 2021
efa24b7
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Aug 24, 2021
e89c332
Add missing update function
JulienIgnace-Unity Aug 25, 2021
8a8b47d
Proper error handling on apv texture allocation failure
JulienIgnace-Unity Aug 25, 2021
c580086
Handle APV index allocation failure
JulienIgnace-Unity Aug 25, 2021
7a7e08e
Small cleanup
JulienIgnace-Unity Aug 25, 2021
a584b9b
Create hidden per scene data
FrancescoC-unity Aug 27, 2021
87fed75
Start using new mode in baking workflow (Need to fix the tmp send data)
FrancescoC-unity Aug 27, 2021
922f827
Remove loading/removal from authoring
FrancescoC-unity Aug 30, 2021
49da9e8
Need to switch branch, move profile data to asset
FrancescoC-unity Aug 31, 2021
7e86ae5
Almost completely get rid of baking reference volume
FrancescoC-unity Aug 31, 2021
a92bf42
remove baking ref from realtime subdiv (but not working :p)
FrancescoC-unity Aug 31, 2021
5221d9b
Precomputed chunk budget for indices and sh pool
JulienIgnace-Unity Aug 31, 2021
fb26eaf
Revert "remove baking ref from realtime subdiv (but not working :p)"
FrancescoC-unity Aug 31, 2021
17fd32e
Remove all probe ref volume authroing from baking, yay!
FrancescoC-unity Aug 31, 2021
809f560
Moving draw gizmo away and remove all reference of authoring from baking
FrancescoC-unity Sep 1, 2021
332ec81
Make sure baking profile is available if required during non baking
FrancescoC-unity Sep 1, 2021
e628d77
Load profile info from the asset
FrancescoC-unity Sep 1, 2021
802547a
Fix dilation refresh
FrancescoC-unity Sep 1, 2021
a59557d
Moved probe pool chunk size constant to ProbeBrickPool
JulienIgnace-Unity Sep 1, 2021
afaf065
Moved internal array to its own class
JulienIgnace-Unity Sep 1, 2021
24fee3a
Removed empty update for now
JulienIgnace-Unity Sep 1, 2021
26328aa
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 1, 2021
ab018ad
Post merge fix
JulienIgnace-Unity Sep 1, 2021
4559ebc
remove some other things
FrancescoC-unity Sep 1, 2021
22d13cf
remove asset
FrancescoC-unity Sep 2, 2021
8f88873
Small rename
FrancescoC-unity Sep 2, 2021
b7c8fc4
Merge branch 'HDRP/new-apv-workflow-backend' of https://github.com/Un…
JulienIgnace-Unity Sep 2, 2021
eb52367
Move field inside EDITOR tags
FrancescoC-unity Sep 2, 2021
520c27c
Post merge fix
JulienIgnace-Unity Sep 2, 2021
1c765d8
Fix issue when no profile is ever loaded (i.e. first time a project i…
FrancescoC-unity Sep 2, 2021
793a308
guards
FrancescoC-unity Sep 2, 2021
a65116c
Merge branch 'HDRP/new-apv-workflow-backend' of https://github.com/Un…
JulienIgnace-Unity Sep 2, 2021
5552a6f
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 3, 2021
e6c7d64
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 6, 2021
6375a26
Streaming V1 (non functional)
JulienIgnace-Unity Sep 8, 2021
1f8526b
Fixed adding the same cells multiple times to the system.
JulienIgnace-Unity Sep 9, 2021
59df65c
Fixed cell sorting
JulienIgnace-Unity Sep 9, 2021
2268168
Fixed wrong unload of unloaded cells.
JulienIgnace-Unity Sep 10, 2021
e2f8bd1
Fixed a mistake in chunk estimation calculation.
JulienIgnace-Unity Sep 10, 2021
50295ab
Moved cell streaing stuff to its own file
JulienIgnace-Unity Sep 13, 2021
467874d
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 13, 2021
7a04814
Merge branch 'hd/apv-cell-streaming' of https://github.com/Unity-Tech…
JulienIgnace-Unity Sep 13, 2021
02e4621
Fixed cell camera distance computation
JulienIgnace-Unity Sep 13, 2021
cdacf4f
GCAlloc hunt 1
JulienIgnace-Unity Sep 14, 2021
771022c
Pool a lot of internal objects to avoid gcallocs
JulienIgnace-Unity Sep 14, 2021
74fd909
Preallocated temporary Color arrays for updating SH textures
JulienIgnace-Unity Sep 14, 2021
933c678
Added various API and tests to DynamicArray
JulienIgnace-Unity Sep 15, 2021
534d049
Quicksort WIP
JulienIgnace-Unity Sep 15, 2021
4b4fcc2
Implemented quicksort
JulienIgnace-Unity Sep 16, 2021
e65b6e6
Reverted wrong change
JulienIgnace-Unity Sep 16, 2021
8fa4822
Moved QuickSort to a class extension and completed doc
JulienIgnace-Unity Sep 16, 2021
c4dfa09
Small improvement to AddRange
JulienIgnace-Unity Sep 16, 2021
be6493f
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 16, 2021
5dd31b7
Post merge fix
JulienIgnace-Unity Sep 16, 2021
d77069c
Update Changelog
JulienIgnace-Unity Sep 16, 2021
9aa5b44
Merge branch 'hd/apv-cell-streaming' of https://github.com/Unity-Tech…
JulienIgnace-Unity Sep 16, 2021
5d2f8df
Merge branch 'hd/dynamic-array-improvements' of https://github.com/Un…
JulienIgnace-Unity Sep 16, 2021
9d56479
Moved internal array to DynamicArray
JulienIgnace-Unity Sep 17, 2021
d162005
Pooled CellInfo to avoid allocation
JulienIgnace-Unity Sep 17, 2021
3cc6639
Split Update of SH texture in several chunks to avoid allocations.
JulienIgnace-Unity Sep 20, 2021
8022bbf
Fixed chunk copy
JulienIgnace-Unity Sep 20, 2021
8f275a6
Fixed Temporary DataLoc lifetime.
JulienIgnace-Unity Sep 20, 2021
c2b8ba4
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 20, 2021
08d6d59
Relax some condition on RemoveRange in DynamicArray
JulienIgnace-Unity Sep 20, 2021
ba00940
Revert MeshGizmo modifications
JulienIgnace-Unity Sep 20, 2021
be16a6a
Restored changed/hidden gizmo when Cells aren't loaded.
JulienIgnace-Unity Sep 20, 2021
287d3fd
Reverted a bit of temp test code.
JulienIgnace-Unity Sep 20, 2021
5e1e4b3
Small Clean
JulienIgnace-Unity Sep 20, 2021
b0197d4
Added a way to disable streaming of cells.
JulienIgnace-Unity Sep 21, 2021
087a401
Fixed hideFlags for pool textures
JulienIgnace-Unity Sep 21, 2021
c81c29d
partial index buffer update to improve performance.
JulienIgnace-Unity Sep 21, 2021
736bfcb
Fixed an issue where cells would stay in internal structures even aft…
JulienIgnace-Unity Sep 21, 2021
7b832eb
Added names to texture allocated for APV
JulienIgnace-Unity Sep 22, 2021
c9f95a3
Removed useless comment
JulienIgnace-Unity Sep 22, 2021
baa9da3
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 22, 2021
adcaa62
Fixed wrong bound for loop
JulienIgnace-Unity Sep 24, 2021
393bc55
Addressed most PR review feedback
JulienIgnace-Unity Sep 24, 2021
6c51d0a
Filled incoming chunks left over space with black.
JulienIgnace-Unity Sep 24, 2021
d30a196
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Sep 27, 2021
9cfac88
Handled the case where not every cell fit in memory when performing d…
JulienIgnace-Unity Oct 7, 2021
0119f3d
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Oct 7, 2021
a532ab5
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Oct 8, 2021
c93e747
Reworked baking to avoid using runtime structure to fill baked assets
JulienIgnace-Unity Oct 8, 2021
8b96607
Moved reference counting into CellInfo
JulienIgnace-Unity Oct 8, 2021
0b2abad
Moved the APV cell streaming options to the HDRP asset.
JulienIgnace-Unity Oct 12, 2021
d70b848
Fixed cleanup of loaded data
JulienIgnace-Unity Oct 12, 2021
18f7ae8
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Oct 18, 2021
a307b05
Merge branch 'master' of https://github.com/Unity-Technologies/Graphi…
JulienIgnace-Unity Oct 20, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Start using new mode in baking workflow (Need to fix the tmp send data)
  • Loading branch information
FrancescoC-unity committed Aug 27, 2021
commit 87fed7508c5360ee5822bb2130d0967c20f264d8
Original file line number Diff line number Diff line change
Expand Up @@ -84,19 +84,12 @@ static void ClearBakingBatch()

static public void Clear()
{
var refVolAuthList = GameObject.FindObjectsOfType<ProbeReferenceVolumeAuthoring>();

foreach (var refVolAuthoring in refVolAuthList)
var perSceneData = GameObject.FindObjectsOfType<ProbeVolumePerSceneData>();
foreach (var data in perSceneData)
{
if (!refVolAuthoring.enabled || !refVolAuthoring.gameObject.activeSelf)
continue;

refVolAuthoring.volumeAsset = null;

data.InvalidateAllAssets();
var refVol = ProbeReferenceVolume.instance;
refVol.Clear();
refVol.SetTRS(Vector3.zero, Quaternion.identity, refVolAuthoring.brickSize);
refVol.SetMaxSubdivision(refVolAuthoring.maxSubdivision);
}

var probeVolumes = GameObject.FindObjectsOfType<ProbeVolume>();
Expand Down Expand Up @@ -279,7 +272,6 @@ internal static void RevertDilation()
{
var refVolAuthList = GameObject.FindObjectsOfType<ProbeReferenceVolumeAuthoring>();
m_BakingReferenceVolumeAuthoring = GetCardinalAuthoringComponent(refVolAuthList);
m_BakingReferenceVolumeAuthoring = GetCardinalAuthoringComponent(refVolAuthList);
if (m_BakingReferenceVolumeAuthoring == null) return;

var dilationSettings = m_BakingReferenceVolumeAuthoring.GetDilationSettings();
Expand All @@ -306,9 +298,9 @@ internal static void RevertDilation()
// proper UX.
internal static void PerformDilation()
{
HashSet<ProbeReferenceVolumeAuthoring> refVols = new HashSet<ProbeReferenceVolumeAuthoring>();
Dictionary<int, List<string>> cell2Assets = new Dictionary<int, List<string>>();
var refVolAuthList = GameObject.FindObjectsOfType<ProbeReferenceVolumeAuthoring>();
var refVolAuthList = GameObject.FindObjectsOfType<ProbeReferenceVolumeAuthoring>(); // TODO: Soon to be removable
var perSceneDataList = GameObject.FindObjectsOfType<ProbeVolumePerSceneData>();

m_BakingReferenceVolumeAuthoring = GetCardinalAuthoringComponent(refVolAuthList);
if (m_BakingReferenceVolumeAuthoring == null) return;
Expand All @@ -317,27 +309,20 @@ internal static void PerformDilation()
{
if (m_BakingReferenceVolumeAuthoring == null)
m_BakingReferenceVolumeAuthoring = refVol;

if (refVol.enabled)
{
refVols.Add(refVol);
}
}

foreach (var refVol in refVols)
foreach (var sceneData in perSceneDataList)
{
if (refVol.volumeAsset != null)
var asset = sceneData.GetCurrentStateAsset();
string assetPath = asset.GetSerializedFullPath();
foreach (var cell in asset.cells)
{
string assetPath = refVol.volumeAsset.GetSerializedFullPath();
foreach (var cell in refVol.volumeAsset.cells)
if (!cell2Assets.ContainsKey(cell.index))
{
if (!cell2Assets.ContainsKey(cell.index))
{
cell2Assets.Add(cell.index, new List<string>());
}

cell2Assets[cell.index].Add(assetPath);
cell2Assets.Add(cell.index, new List<string>());
}

cell2Assets[cell.index].Add(assetPath);
}
}

Expand Down Expand Up @@ -366,11 +351,13 @@ internal static void PerformDilation()
dilatedCells.Add(cell);
}

foreach (var refVol in refVols)
foreach (var sceneData in perSceneDataList)
{
if (refVol != null && refVol.volumeAsset != null)
var asset = sceneData.GetCurrentStateAsset();
string assetPath = asset.GetSerializedFullPath();
if (asset != null)
{
ProbeReferenceVolume.instance.AddPendingAssetRemoval(refVol.volumeAsset);
ProbeReferenceVolume.instance.AddPendingAssetRemoval(asset);
}
}

Expand All @@ -381,11 +368,12 @@ internal static void PerformDilation()
// Put back cells
foreach (var cell in dilatedCells)
{
foreach (var refVol in refVols)
foreach (var sceneData in perSceneDataList)
{
if (refVol.volumeAsset == null) continue;
var asset = sceneData.GetCurrentStateAsset();

if (asset == null) continue;

var asset = refVol.volumeAsset;
var assetPath = asset.GetSerializedFullPath();
bool valueFound = false;
if (!assetCleared.TryGetValue(assetPath, out valueFound))
Expand All @@ -404,10 +392,9 @@ internal static void PerformDilation()
UnityEditor.AssetDatabase.SaveAssets();
UnityEditor.AssetDatabase.Refresh();

foreach (var refVol in refVols)
foreach (var sceneData in perSceneDataList)
{
if (refVol.enabled && refVol.gameObject.activeSelf)
refVol.QueueAssetLoading();
sceneData.QueueAssetLoading();
}
}

Expand Down Expand Up @@ -534,16 +521,15 @@ static void OnAdditionalProbesBakeCompleted()
UnityEditor.Experimental.Lightmapping.SetAdditionalBakedProbes(m_BakingBatch.index, null);

// Map from each scene to an existing reference volume
var scene2RefVol = new Dictionary<Scene, ProbeReferenceVolumeAuthoring>();
foreach (var refVol in GameObject.FindObjectsOfType<ProbeReferenceVolumeAuthoring>())
if (refVol.enabled)
scene2RefVol[refVol.gameObject.scene] = refVol;
var scene2Data = new Dictionary<Scene, ProbeVolumePerSceneData>();
foreach (var data in GameObject.FindObjectsOfType<ProbeVolumePerSceneData>())
scene2Data[data.gameObject.scene] = data;

// Map from each reference volume to its asset
var refVol2Asset = new Dictionary<ProbeReferenceVolumeAuthoring, ProbeVolumeAsset>();
foreach (var refVol in scene2RefVol.Values)
var data2Asset = new Dictionary<ProbeVolumePerSceneData, ProbeVolumeAsset>();
foreach (var data in scene2Data.Values)
{
refVol2Asset[refVol] = ProbeVolumeAsset.CreateAsset(refVol.gameObject.scene);
data2Asset[data] = ProbeVolumeAsset.CreateAsset(data.gameObject.scene);
}

// Put cells into the respective assets
Expand All @@ -552,29 +538,32 @@ static void OnAdditionalProbesBakeCompleted()
foreach (var scene in m_BakingBatch.cellIndex2SceneReferences[cell.index])
{
// This scene has a reference volume authoring component in it?
ProbeReferenceVolumeAuthoring refVol = null;
if (scene2RefVol.TryGetValue(scene, out refVol))
ProbeVolumePerSceneData data = null;
if (scene2Data.TryGetValue(scene, out data))
{
var asset = refVol2Asset[refVol];
var asset = data2Asset[data];
asset.cells.Add(cell);
CellCountInDirections(out asset.minCellPosition, out asset.maxCellPosition, refVol.profile.cellSizeInMeters);
var profile = ProbeReferenceVolume.instance.sceneData.GetProfileForScene(scene);
Debug.Assert(profile != null);
CellCountInDirections(out asset.minCellPosition, out asset.maxCellPosition, profile.cellSizeInMeters);
asset.globalBounds = globalBounds;
}
}
}

// Connect the assets to their components
foreach (var pair in refVol2Asset)
foreach (var pair in data2Asset)
{
var refVol = pair.Key;
var data = pair.Key;
var asset = pair.Value;

refVol.volumeAsset = asset;
// TODO: This will need to use the proper state, not default, when we have them.
data.StoreAssetForState(ProbeVolumeState.Default, asset);

if (UnityEditor.Lightmapping.giWorkflowMode != UnityEditor.Lightmapping.GIWorkflowMode.Iterative)
{
UnityEditor.EditorUtility.SetDirty(refVol);
UnityEditor.EditorUtility.SetDirty(refVol.volumeAsset);
UnityEditor.EditorUtility.SetDirty(data);
UnityEditor.EditorUtility.SetDirty(asset);
}
}

Expand All @@ -588,10 +577,9 @@ static void OnAdditionalProbesBakeCompleted()
UnityEditor.AssetDatabase.Refresh();
probeRefVolume.clearAssetsOnVolumeClear = false;

foreach (var refVol in refVol2Asset.Keys)
foreach (var data in data2Asset.Keys)
{
if (refVol.enabled && refVol.gameObject.activeSelf)
refVol.QueueAssetLoading();
data.QueueAssetLoading();
}

// ---- Perform dilation ---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,14 @@ internal void LoadProfileInformation()
refVol.dilationValidtyThreshold = m_DilationValidityThreshold;
}

// TEMP! THIS NEEDS TO BE REMOVED WHEN WE HAVE THE SETTINGS PANEL.
void SendSceneData()
{
if (ProbeReferenceVolume.instance.sceneData == null) return;
ProbeReferenceVolume.instance.sceneData.SetBakeSettingsForScene(gameObject.scene, GetDilationSettings(), GetVirtualOffsetSettings());
ProbeReferenceVolume.instance.sceneData.SetProfileForScene(gameObject.scene, m_Profile);
}

internal void QueueAssetLoading()
{
LoadProfileInformation();
Expand Down Expand Up @@ -163,6 +171,7 @@ void OnEnable()
if (m_Profile == null)
m_Profile = CreateReferenceVolumeProfile(gameObject.scene, gameObject.name);
#endif
SendSceneData();
QueueAssetLoading();
}

Expand All @@ -189,6 +198,7 @@ void OnValidate()
}

m_PrevAsset = volumeAsset;
SendSceneData();
}

void OnDisable()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
using UnityEngine.Rendering;
#if UNITY_EDITOR
using UnityEditor;
#endif

namespace UnityEngine.Experimental.Rendering
{
// TODO: Use this structure in the actual authoring component rather than just a mean to group output parameters.
internal struct ProbeVolumeBakingProcessSettings
{
public ProbeDilationSettings dilationSettings;
public VirtualOffsetSettings virtualOffsetSettings;
}
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,28 @@ public void OnBeforeSerialize()
}
}

internal void StoreAssetForState(ProbeVolumeState state, ProbeVolumeAsset asset)
{
assets[state] = asset;
}

internal void InvalidateAllAssets()
{
foreach (var asset in assets.Values)
{
if (asset != null)
ProbeReferenceVolume.instance.AddPendingAssetRemoval(asset);
}

assets.Clear();
}

internal ProbeVolumeAsset GetCurrentStateAsset()
{
if (assets.ContainsKey(m_CurrentState)) return assets[m_CurrentState];
else return null;
}

internal void QueueAssetLoading()
{
if (assets.ContainsKey(m_CurrentState) && assets[m_CurrentState] != null)
Expand Down
Loading