Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Visible blocking defects in slow-moving objects #489

Open
e8m-8 opened this issue Dec 12, 2024 · 0 comments
Open

Visible blocking defects in slow-moving objects #489

e8m-8 opened this issue Dec 12, 2024 · 0 comments

Comments

@e8m-8
Copy link

e8m-8 commented Dec 12, 2024

As the QP value increases, the blocking defects around the edges of the moving object with noticeable changes become more obvious. When the motion is slow, the defects are very visible even if I don't pause.

To mitigate this problem, I modified the weights of va.hpTempAct based on m_QP.

  double qpFactor = (m_QP < 20) ? 2.0 :
                    (m_QP < 30) ? (20.0 / double(m_QP - 9)) :
                    (m_QP < 40) ? ((0.212 - 0.004 * m_QP) * m_QP - 1.784) :
                                  0.4;
  va.hpVisAct = std::max(double(1 << (bitDepth - 6)), va.hpSpatAct + qpFactor * va.hpTempAct);

These weights are based on traversing the coded of Adaptive QP from 15 to 45 after each modification of my qpFactor and were obtained after VMAF, SSIM detection plus actual viewing inspection.
I think XPSNR is extremely valid, but motion defects at high QP may not be an issue with XPSNR. It's just a solution I'm currently using with VVC.
Even if I have to move the qpFactor, in my opinion it should be in a small range of intervals like 1.0 to 2.0, otherwise the point of XPSNR may be lost.

For example when I coded an clean 1920x1080 25fps 10bit 4:2:0 Y4M by Adaptive QP39:
vvencFFapp -i MQPA-PreRAW_10_420.y4m --Preset slow --Level 5.1 --FrameRate 25 --QP 39 --PerceptQPA 1 --IntraPeriod 250 --GOPSize 32 --MinIntraDistance 1 -b MQPA-39.266

e.g. In a long shot, the girl rides a wooden horse that rotates sideways to the front as the camera slowly pulls up.
Results from programmes compiled using different weights:

va.hpVisAct = std::max(double(1 << (bitDepth - 6)), va.hpSpatAct + 2.0 * va.hpTempAct)
QPA39-20241212

va.hpVisAct = std::max(double(1 << (bitDepth - 6)), va.hpSpatAct + 1.0 * va.hpTempAct)
QPA39-hpTempAct1-241212

va.hpVisAct = std::max(double(1 << (bitDepth - 6)), va.hpSpatAct + 0.4 * va.hpTempAct)
QPA39-hpTempAct0 4-241212

Mitigation, not solution.
Defects are reduced to visually negligible when 0.4 * va.hpTempAct in QPA39.
Random blocks still appear to surround the edges.

Here's my entire project change, based on vvenc-v1.13.0-rc1 before merging the ARM optimisation:

diff --git a/source/Lib/EncoderLib/BitAllocation.cpp b/source/Lib/EncoderLib/BitAllocation.cpp
index 13f2ede..be6d063 100644
--- a/source/Lib/EncoderLib/BitAllocation.cpp
+++ b/source/Lib/EncoderLib/BitAllocation.cpp
@@ -194,17 +194,22 @@ void calcTemporalVisAct ( const Pel* pSrc,
   va.tempAct = unsigned (0.5 + va.hpTempAct * double (bitDepth < 12 ? 1 << (12 - bitDepth) : 1) * (frameRate <= 31 ? 1.15625 : 1.0));
 }

-void updateVisAct ( VisAct& va, const uint32_t bitDepth )
+void updateVisAct ( VisAct& va, const uint32_t bitDepth, const int m_QP)
 {
   // minimum part in 12 bit
   va.minAct = std::min( va.tempAct, va.spatAct );
   // lower limit, compensate for high-pass amplification
-  va.hpVisAct = std::max (double (1 << (bitDepth - 6)), va.hpSpatAct + 2.0 * va.hpTempAct);
+  double qpFactor = (m_QP < 20) ? 2.0 :
+                    (m_QP < 30) ? (20.0 / double(m_QP - 9)) :
+                    (m_QP < 40) ? ((0.212 - 0.004 * m_QP) * m_QP - 1.784) :
+                                  0.4;
+  va.hpVisAct = std::max(double(1 << (bitDepth - 6)), va.hpSpatAct + qpFactor * va.hpTempAct);
   va.visAct   = ClipBD( uint16_t( 0.5 + va.hpVisAct ), bitDepth );
 }

 double filterAndCalculateAverageActivity ( const Pel* pSrc,
                                            const int iSrcStride,
+                                           const VVEncCfg* encCfg,
                                            const int height,
                                            const int width,
                                            const Pel* pSM1,
@@ -227,7 +232,7 @@ double filterAndCalculateAverageActivity ( const Pel* pSrc,
                      frameRate, bitDepth, isUHD, va );

   // minimum and visual activity
-  updateVisAct( va, bitDepth );
+  updateVisAct( va, bitDepth, encCfg->m_QP);

   if( minVisAct )
   {
@@ -562,7 +567,7 @@ int BitAllocation::applyQPAdaptationSlice (const Slice* slice, const VVEncCfg* e
         const CPelBuf  picPrv2   = pic->getOrigBufPrev (fltArea, PREV_FRAME_2);
         unsigned minActivityPart = 0, spVisActCTU = 0;

-        hpEner[1] = filterAndCalculateAverageActivity (picOrig.buf, picOrig.stride, picOrig.height, picOrig.width,
+        hpEner[1] = filterAndCalculateAverageActivity (picOrig.buf, picOrig.stride, encCfg, picOrig.height, picOrig.width,
                                                        picPrv1.buf, picPrv1.stride, picPrv2.buf, picPrv2.stride, hpFrameRate,
                                                        bitDepth, isHighResolution, &minActivityPart, &spVisActCTU);

@@ -611,7 +616,7 @@ int BitAllocation::applyQPAdaptationSlice (const Slice* slice, const VVEncCfg* e
       const CPelBuf picPrv1 = pic->getOrigBufPrev (compID, PREV_FRAME_1);
       const CPelBuf picPrv2 = pic->getOrigBufPrev (compID, PREV_FRAME_2);

-      hpEner[comp] = filterAndCalculateAverageActivity (picOrig.buf, picOrig.stride, picOrig.height, picOrig.width,
+      hpEner[comp] = filterAndCalculateAverageActivity (picOrig.buf, picOrig.stride, encCfg, picOrig.height, picOrig.width,
                                                         picPrv1.buf, picPrv1.stride, picPrv2.buf, picPrv2.stride, hpFrameRate,
                                                         bitDepth, isHighResolution && (pic->chromaFormat == CHROMA_444));

@@ -835,7 +840,7 @@ int BitAllocation::applyQPAdaptationSubCtu (const Slice* slice, const VVEncCfg*
   const CPelBuf     picOrig   = pic->getOrigBuf (fltArea);
   const CPelBuf     picPrv1   = pic->getOrigBufPrev (fltArea, PREV_FRAME_1);
   const CPelBuf     picPrv2   = pic->getOrigBufPrev (fltArea, PREV_FRAME_2);
-  const double hpEnerSubCTU   = filterAndCalculateAverageActivity (picOrig.buf, picOrig.stride, picOrig.height, picOrig.width,
+  const double hpEnerSubCTU   = filterAndCalculateAverageActivity (picOrig.buf, picOrig.stride, encCfg, picOrig.height, picOrig.width,
                                                                    picPrv1.buf, picPrv1.stride, picPrv2.buf, picPrv2.stride, hpFrameRate,
                                                                    bitDepth, isHighResolution);
   const double hpEnerPicNorm  = 1.0 / getAveragePictureActivity (encCfg->m_SourceWidth, encCfg->m_SourceHeight, 0,

diff --git a/source/Lib/EncoderLib/BitAllocation.h b/source/Lib/EncoderLib/BitAllocation.h
index 004bae6..2363028 100644
--- a/source/Lib/EncoderLib/BitAllocation.h
+++ b/source/Lib/EncoderLib/BitAllocation.h
@@ -97,10 +97,11 @@ namespace vvenc {
                             const bool isUHD,
                             VisAct& va );

-  void updateVisAct ( VisAct& va, const uint32_t bitDepth );
+  void updateVisAct ( VisAct& va, const uint32_t bitDepth, const int m_QP);

   double filterAndCalculateAverageActivity ( const Pel* pSrc,
                                              const int iSrcStride,
+                                             const VVEncCfg* encCfg,
                                              const int height,
                                              const int width,
                                              const Pel* pSM1,

diff --git a/source/Lib/EncoderLib/PreProcess.cpp b/source/Lib/EncoderLib/PreProcess.cpp
index 952c4b8..ebd1196 100644
--- a/source/Lib/EncoderLib/PreProcess.cpp
+++ b/source/Lib/EncoderLib/PreProcess.cpp
@@ -340,7 +340,7 @@ void PreProcess::xGetVisualActivity( Picture* pic, const PicList& picList ) cons
     xGetPrevPics( pic, picList, prevPics );
     xGetTemporalActivity( pic, prevPics[ 0 ], prevPics[ 1 ], va[ CH_L ] );
     // visual activity for picture
-    updateVisAct( va[ CH_L ], m_encCfg->m_internalBitDepth[ CH_L ] );
+    updateVisAct( va[ CH_L ], m_encCfg->m_internalBitDepth[ CH_L ], m_encCfg->m_QP );
   }

   // visual activity to previous TL0 pic (luma only)
@@ -350,7 +350,7 @@ void PreProcess::xGetVisualActivity( Picture* pic, const PicList& picList ) cons
     // get temporal activity for picture
     xGetTemporalActivity( pic, prevTL0, nullptr, vaTL0 );
     // visual activity for picture
-    updateVisAct( vaTL0, m_encCfg->m_internalBitDepth[ CH_L ] );
+    updateVisAct( vaTL0, m_encCfg->m_internalBitDepth[ CH_L ], m_encCfg->m_QP );
   }

   // store visual activity

Many thanks to all the vvenc contributors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant