Repository navigation
Send pull replications to the location pulled from, with the destinat… - #2931
SylvainSenechal wants to merge 1 commit into
Conversation
…ion in pullDestination Issue: BB-913
Hello sylvainsenechal,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
| */ | ||
| getCopyDestination() { | ||
| return this.isPullReplication() ? | ||
| this.getAttribute('pullDestination') : this.getAttribute('toLocation'); |
There was a problem hiding this comment.
A pull replication action without pullDestination (e.g. still in Kafka from before this change, where toLocation was the destination) returns undefined here. LifecycleUpdateTransitionTask would then write dataStoreName/storage class as undefined into the object metadata. Fall back to toLocation:
| this.getAttribute('pullDestination') : this.getAttribute('toLocation'); | |
| (this.getAttribute('pullDestination') ?? this.getAttribute('toLocation')) : | |
| this.getAttribute('toLocation'); |
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (32.09%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files
@@ Coverage Diff @@
## development/9.7 #2931 +/- ##
===================================================
- Coverage 81.24% 81.15% -0.10%
===================================================
Files 209 209
Lines 44290 44342 +52
===================================================
+ Hits 35985 35986 +1
- Misses 8305 8356 +51
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
- can't rely on 'toLocation' anymore, need to go through the helper getCopyDestination to check if we are in a pullReplication context
- New Kafka message field pullDestination, used in that new getCopyDestination helper because we still need to know the destination that isn't passed in toLocation anymore but in pullDestination now
- we still need that new _getDestination helper everywhere to decide which location to write to
- Potentially misleading message logs (at least one spotted, fixable but some more if else thing)
But compared with the other pr, we don't have the "siteInCharge" part
const siteInCharge = actionEntry.isPullReplication() ?
actionEntry.getAttribute('fromLocation') : actionEntry.getAttribute('toLocation');
Ater looking again at both pr, I think the first one is less complex than this one, I would keep the first pr, unless we absolutely do not want to have the data processor invert the copy data
| objMD.getTransitionTime(); | ||
| LifecycleMetrics.onLifecycleStarted(log, getCopyLocationMetricsType(actionEntry), | ||
| actionEntry.getAttribute('toLocation'), | ||
| actionEntry.getCopyDestination(), |
There was a problem hiding this comment.
should not be needed?
| actionEntry.getAttribute('metrics'); | ||
| ReplicationMetrics.onReplicationProcessed( | ||
| origin, fromLocation, this.site, contentLength, | ||
| origin, fromLocation, this._getDestination(actionEntry).location, contentLength, |
There was a problem hiding this comment.
should not be needed?
| // fields, shared with cloudserver | ||
| const { location } = entry.getAttribute('results'); | ||
| const newLocationName = entry.getAttribute('toLocation'); | ||
| const newLocationName = entry.getCopyDestination(); |
There was a problem hiding this comment.
instead of tweaking LifecycleUpdateTransitionTask, should be handled in CopyLocationTask::_publishCopyLocationStatus
| StorageType: storageType, | ||
| StorageClass: location, |
There was a problem hiding this comment.
ok, I think this maybe the critical point
- we pass the storageClass/Type in many places : so this makes the approach as risky as the other one...
- ...but actually we don't need to change these places : these commands are only used in actual transition, and not in the pull-replication case
There was a problem hiding this comment.
- ...but actually we don't need to change these places : these commands are only used in actual transition, and not in the pull-replication case
sorry, I mixed things up. This is indeed the write on the (real) "destination" data location, and must be fixed indeed.
| transitionTime: params.transitionTime, | ||
| }); | ||
| if (params.pullDestination) { | ||
| action.setAttribute('pullDestination', params.pullDestination); |
There was a problem hiding this comment.
- this is used by
LifecycleUpdateTransitionTask.js, notCopyLocationTask: so not needed - in copyLocationTask, the actual "target" storageClass is already available in the location pointer
Issue: BB-913
Draft to compare with #2876