Limit dotProduct value to 1.0f, so acosf would not return NaN.

Cherry pick of b561f39d01c211425bfefaaa7b31ebe097e7ba79 from AOSP master.

Due to precision loss of float math, we sometimes get 1.000001f for
dotProduct. This causes NaN result from acosf() and floor() funcs.

At the moment, this does not cause any problems on ARM, as casting
NaN to int results in 0. On mips however (possibly on x86), such cast
gives INT_MAX, so crash occurs when trying to use the resulting value.

Change-Id: I8e0285a0306a65b8469d9f4885c19665066fc4c8

1 files changed, 3 insertions, 0 deletions

diff --git a/libs/hwui/ShadowTessellator.cpp b/libs/hwui/ShadowTessellator.cpp
index 024ff10..09d1258 100644
--- a/libs/hwui/ShadowTessellator.cpp
+++ b/libs/hwui/ShadowTessellator.cpp
@@ -21,6 +21,7 @@
 #include <utils/Log.h>
 #include <utils/Trace.h>
 #include <utils/Vector.h>
+#include <utils/MathUtils.h>
 
 #include "AmbientShadow.h"
 #include "Properties.h"
@@ -172,6 +173,8 @@ int ShadowTessellator::getExtraVertexNumber(const Vector2& vector1,
     // acos( )     --- [0, M_PI]
     // floor(...)  --- [0, EXTRA_VERTEX_PER_PI]
     float dotProduct = vector1.dot(vector2);
+    // make sure that dotProduct value is in acsof input range [-1, 1]
+    dotProduct = MathUtils::clamp(dotProduct, -1.0f, 1.0f);
     // TODO: Use look up table for the dotProduct to extraVerticesNumber
     // computation, if needed.
     float angle = acosf(dotProduct);