0
0
之前已经针对EVS的整体架构进行了解析，并对各个模块的功能进行了初步的介绍和讲解，接下来我们将深入到每个模块的内部，看看模块内的具体逻辑，那么今天我们解析EVS架构中的应用——EVS APP。
首先还是需要说明，在Android系统中谈到App很多人都默认其是运行在JVM之上的Application，但是此处的EVS APP则是运行在Native Framework这一层的，非传统Android基于apk形式的应用。之所以称之为EVS APP，是因为其具体的功能逻辑，与运行在JVM上的APP保持一致，即基于用户需求，响应用户输入，实现用户交互。
程序入口
EVS APP的入口我们可以在evs_app.cpp文件内找到，我们找到main程序入口：
// Main entry point
int main(int argc, char** argv)
{
LOG(INFO) << "EVS app starting";
// Register a signal handler
registerSigHandler();
// Set up default behavior, then check for command line options
bool useVehicleHal = true;
bool printHelp = false;
const char* evsServiceName = "default";
int displayId = -1;
bool useExternalMemory = false;
android_pixel_format_t extMemoryFormat = HAL_PIXEL_FORMAT_RGBA_8888;
int32_t mockGearSignal = static_cast<int32_t>(VehicleGear::GEAR_REVERSE);
for (int i = 1; i < argc; i++)
{
...
}
// Load our configuration information
ConfigManager config;
if (!config.initialize(CONFIG_OVERRIDE_PATH))
{
if (!config.initialize(CONFIG_DEFAULT_PATH))
{
return EXIT_FAILURE;
}
}
configureRpcThreadpool(1, false /* callerWillJoin */);
// Construct our async helper object
sp<EvsVehicleListener> pEvsListener = new EvsVehicleListener();
// Get the EVS manager service
LOG(INFO) << "Acquiring EVS Enumerator";
pEvs = IEvsEnumerator::getService(evsServiceName);
if (pEvs.get() == nullptr)
{
LOG(ERROR) << "getService(" << evsServiceName
<< ") returned NULL. Exiting.";
return EXIT_FAILURE;
}
// Request exclusive access to the EVS display
LOG(INFO) << "Acquiring EVS Display";
// We'll use an available display device.
displayId = config.setActiveDisplayId(displayId);
if (displayId < 0)
{
PLOG(ERROR) << "EVS Display is unknown. Exiting.";
return EXIT_FAILURE;
}
pDisplay = pEvs->openDisplay_1_1(displayId);
if (pDisplay.get() == nullptr)
{
LOG(ERROR) << "EVS Display unavailable. Exiting.";
return EXIT_FAILURE;
}
config.useExternalMemory(useExternalMemory);
config.setExternalMemoryFormat(extMemoryFormat);
// Set a mock gear signal for the test mode
config.setMockGearSignal(mockGearSignal);
// Connect to the Vehicle HAL so we can monitor state
sp<IVehicle> pVnet;
if (useVehicleHal)
{
LOG(INFO) << "Connecting to Vehicle HAL";
pVnet = IVehicle::getService();
if (pVnet.get() == nullptr)
{
LOG(ERROR) << "Vehicle HAL getService returned NULL. Exiting.";
return EXIT_FAILURE;
}
else
{
if (!subscribeToVHal(pVnet, pEvsListener, VehicleProperty::GEAR_SELECTION))
{
LOG(ERROR) << "Without gear notification, we can't support EVS. Exiting.";
return EXIT_FAILURE;
}
if (!subscribeToVHal(pVnet, pEvsListener, VehicleProperty::TURN_SIGNAL_STATE))
{
LOG(WARNING) << "Didn't get turn signal notifications, so we'll ignore those.";
}
}
}
else
{
LOG(WARNING) << "Test mode selected, so not talking to Vehicle HAL";
}
// Configure ourselves for the current vehicle state at startup
LOG(INFO) << "Constructing state controller";
pStateController = new EvsStateControl(pVnet, pEvs, pDisplay, config);
if (!pStateController->startUpdateLoop())
{
LOG(ERROR) << "Initial configuration failed. Exiting.";
return EXIT_FAILURE;
}
// Run forever, reacting to events as necessary
LOG(INFO) << "Entering running state";
pEvsListener->run(pStateController);
// In normal operation, we expect to run forever, but in some error conditions we'll quit.
// One known example is if another process preempts our registration for our service name.
LOG(ERROR) << "EVS Listener stopped. Exiting.";
return EXIT_SUCCESS;
}在main函数内，主要做了以下几件事：
1.注册信号处理函数，如进程收到SIGABRT、SIGTERM与SIGINT等信号后会向EvsStateControl模块发送命令结束内部线程。
2.解析程序启动参数，如display设定、车辆挡位设定等信号；如果未匹配到对应的启动参数则打印帮助信息，这部分一般用于调试。
3.解析配置文件，配置文件用于配置应用所需要的一些信息
4.获取底层Hal枚举对象，打开Display设备
5.向Vehicle注册订阅信号，并创建EvsVehicleListener来对回调做处理；这里只订阅了挡位和转向灯信息
6.启动StateControl模块内的loop线程
开启绘制
在EVS架构中，需要一直保持对车身信息的监听，从而来决定是否需要进行画面的绘制，以及切换画面的状态。而这一切控制的源头，来源于EvsStateControl模块：
void EvsStateControl::updateLoop()
{
ALOGD("Starting EvsStateControl update loop");
bool run = true;
while (run)
{
// Process incoming commands
{
std::lock_guard<std::mutex> lock(mLock);
while (!mCommandQueue.empty())
{
const Command& cmd = mCommandQueue.front();
switch (cmd.operation)
{
case Op::EXIT:
run = false;
break;
case Op::CHECK_VEHICLE_STATE:
break;
case Op::TOUCH_EVENT:
break;
}
mCommandQueue.pop();
}
}
}
...
if (!selectStateForCurrentConditions())
{
ALOGE("selectStateForCurrentConditions failed,loop again");
continue;
}
...
if (!mCurrentRenderer->drawFrame(convertBufferDesc(tgtBuffer)))
{
run = false;
}
// Send the finished image back for display
displayHandle->returnTargetBufferForDisplay(tgtBuffer);
if (!mFirstFrameIsDisplayed)
{
mFirstFrameIsDisplayed = true;
mEvsStats.finishComputingFirstFrameLatency(android::uptimeMillis());
}
...
}在该loop循环中，主要包含三个行为：
1.接收处理三类command:退出，车身状态变化与Touch事件。当前实现中退出的command只在程序捕获到诸如SIGTERM等信号时发送，车身状态变化则在EvsVehicleListener接收到Vehicle HAL回调时进行发送，而Touch事件当前没有实现，是需要开发者自行实现的。
2.根据车身状态选择不同的绘制类型与方式。这里的类型分为两类：RenderDirectView与RenderTopView,这两类绘制类型都有单独的类实现。简单来讲，RenderDirectView是针对于只有单个摄像头图像输入的情况，而RenderTopView则是存在多个摄像头图像输入时的绘制。而绘制方式则是分为GPU加速与纯CPU处理两类。前者需要借助于OpenGL ES实现图像绘制，而后者则是通过CPU直接操作内存。
在EVS架构中，决定绘制的类型由当前的车身状态与配置文件中的配置参数共同决定。车身状态分为：OFF、REVERSE、RIGHT、LEFT、PARKING这五类，REVERSE与PARKING由挡位决定，而RIGHT与LEFT由转向灯决定，挡位的优先级是高于转向灯的。
在得到上述状态后，EVS内会判断配置了上述状态的摄像头个数，当只有一个摄像头时则会以RenderDirectView类型进行图像绘制，而存在多个摄像头时则会议RenderTopView类型进行图像绘制，如下所示。
if (mCameraList[desiredState].size() == 1)
{
mDesiredRenderer =std::make_unique<RenderDirectView>(mEvs,
mCameraDescList[desiredState][0],mConfig);
if (!mDesiredRenderer)
{
LOGE("Failed to construct direct renderer. Skipping state change.");
return false;
}
}
else if (mCameraList[desiredState].size() > 1 || desiredState == PARKING)
{
ConfigManager.mDesiredRenderer =
std::make_unique<RenderTopView>(mEvs, mCameraList[desiredState], mConfig);
if (!mDesiredRenderer)
{
LOGE("Failed to construct top view renderer. Skipping state change.");
return false;
}
}获取Display Buffer
在我们开始正式的绘制时，我们需要一块儿display buffer用于承载我们绘制好的内容用于显示，这些buffer统一都来源于GraphicBuffer。在Android12.1原生EVS实现中，EVS APP通过HIDL接口向EVS HAL申请这类Buffer，定义如下：
/**
* This call returns a handle to a frame buffer associated with the display.
*
* @return buffer A handle to a frame buffer. The returned buffer may be
* locked and written to by software and/or GL. This buffer
* must be returned via a call to
* returnTargetBufferForDisplay() even if the display is no
* longer visible.
*/
getTargetBuffer() generates (BufferDesc buffer);当EVS APP首次通过该接口向EVS HAL申请用于显示的frame buffer时,EVS HAL即会通过GraphicBufferAllocator的服务分配这样一块儿内存，代码如下：
/**
* This call returns a handle to a frame buffer associated with the display.
* This buffer may be locked and written to by software and/or GL. This buffer
* must be returned via a call to returnTargetBufferForDisplay() even if the
* display is no longer visible.
*/
Return<void> EvsGlDisplay::getTargetBuffer(getTargetBuffer_cb _hidl_cb)
{
// ALOGD("%s Entered", __FUNCTION__);
std::lock_guard<std::mutex> lock(mAccessLock);
if (mRequestedState == EvsDisplayState::DEAD)
{
ALOGE("Rejecting buffer request from object that lost ownership of the display.");
_hidl_cb({});
return Void();
}
// If we don't already have a buffer, allocate one now
if (!mBuffer.memHandle)
{
// Initialize our display window
// NOTE: This will cause the display to become "VISIBLE" before a frame is actually
// returned, which is contrary to the spec and will likely result in a black frame being
// (briefly) shown.
if (!mGlWrapper.initialize(mDisplayProxy, mDisplayId))
{
// Report the failure
ALOGE("Failed to initialize GL display");
_hidl_cb({});
return Void();
}
// Assemble the buffer description we'll use for our render target
mBuffer.width = mGlWrapper.getWidth();
mBuffer.height = mGlWrapper.getHeight();
mBuffer.format = HAL_PIXEL_FORMAT_RGBA_8888;
mBuffer.usage = GRALLOC_USAGE_HW_RENDER | GRALLOC_USAGE_HW_COMPOSER;
mBuffer.bufferId = 0x3870; // Arbitrary magic number for self recognition
mBuffer.pixelSize = 4;
// Allocate the buffer that will hold our displayable image
buffer_handle_t handle = nullptr;
GraphicBufferAllocator& alloc(GraphicBufferAllocator::get());
status_t result = alloc.allocate(mBuffer.width, mBuffer.height,
mBuffer.format, 1,
mBuffer.usage, &handle,
&mBuffer.stride,
0, "EvsGlDisplay");
if (result != NO_ERROR)
{
// ALOGE << "Error " << result
// << " allocating " << mBuffer.width << " x " << mBuffer.height
// << " graphics buffer.";
ALOGE("allocating width:%d,height:%d graphics buffer error", mBuffer.width, mBuffer.height);
_hidl_cb({});
mGlWrapper.shutdown();
return Void();
}
if (!handle)
{
ALOGE("We didn't get a buffer handle back from the allocator");
_hidl_cb({});
mGlWrapper.shutdown();
return Void();
}
mBuffer.memHandle = handle;
ALOGD("Allocated new buffer %p with stride:%d ", mBuffer.memHandle.getNativeHandle(), mBuffer.stride);
mFrameBusy = false;
}
// Do we have a frame available?
if (mFrameBusy)
{
// This means either we have a 2nd client trying to compete for buffers
// (an unsupported mode of operation) or else the client hasn't returned
// a previously issued buffer yet (they're behaving badly).
// NOTE: We have to make the callback even if we have nothing to provide
ALOGE("getTargetBuffer called while no buffers available.");
_hidl_cb({});
return Void();
}
else
{
// Mark our buffer as busy
mFrameBusy = true;
// Send the buffer to the client
ALOGV("Providing display buffer handle:%p as id %d ", mBuffer.memHandle.getNativeHandle(), mBuffer.bufferId);
_hidl_cb(mBuffer);
return Void();
}
}
获取Camera Buffer
除了Display Buffer，我们还需要获取Camera数据，涉及到的接口包括：deliverFrame，newFrameAvailable,getNewFrame与doneWithFrame.其中deliverFrame与doneWithFrame都属于HIDL接口，其定义下：
IEvsCamera.hal：
/**
* Return a frame that was delivered by to the IEvsCameraStream.
*
* When done consuming a frame delivered to the IEvsCameraStream
* interface, it must be returned to the IEvsCamera for reuse.
* A small, finite number of buffers are available (possibly as small
* as one), and if the supply is exhausted, no further frames may be
* delivered until a buffer is returned.
*
* @param buffer A buffer to be returned.
*/
oneway doneWithFrame(BufferDesc buffer);IEvsCameraStream.hal：
/**
* Receives calls from the HAL each time a video frame is ready for inspection.
* Buffer handles received by this method must be returned via calls to
* IEvsCamera::doneWithFrame(). When the video stream is stopped via a call
* to IEvsCamera::stopVideoStream(), this callback may continue to happen for
* some time as the pipeline drains. Each frame must still be returned.
* When the last frame in the stream has been delivered, a NULL bufferHandle
* must be delivered, signifying the end of the stream. No further frame
* deliveries may happen thereafter.
*
* @param buffer a buffer descriptor of a delivered image frame.
*/
oneway deliverFrame(BufferDesc buffer);接口执行流程上，当EVS HAL获取到新的帧时，会通过deliverFrame接口将数据上报给EVS APP，EVS APP会将其存储至一个双Buffer中（一个用于存储新的camera数据，一个用于存储当前正在使用中的camera数据，两者循环），在EVS APP绘制时，先通过newFrameAvailable接口判断当前是否有新的数据到来，如存在新的数据，则通过getNewFrame来获取新的camera数据，在使用OpenGL ES接口进行绘制渲染后，再通过 doneWithFrame 将buffer归还给底层。
绘制流程
无论是RenderDirectView还是RenderTopView，其绘制流程大体都是类似的。绘制的控制在EvsStateControl模块内，如下所示：
mCurrentRenderer = std::move(mDesiredRenderer);
// Start the camera stream render
LOGD("EvsStartCameraStreamTiming start time:%lld ms.", android::elapsedRealtime());
if (!mCurrentRenderer->activate())
{
LOGE("New renderer failed to activate");
return false;
}
// Activate the display
LOGD("EvsStartCameraStreamTiming start time:%lld ms.", android::elapsedRealtime());
Return<EvsResult> result =
displayHandle->setDisplayState(EvsDisplayState::VISIBLE_ON_NEXT_FRAME);
if (result != EvsResult::OK)
{
LOGE("setDisplayState returned an error:%s ", result.description().c_str());
return false;
}我们通过调用activate函数来启动Render的过程，这部分涉及到OpenGL ES的一些知识。简单描述一下流程：
1）初始化EGL环境，包括设定eglDisplay、eglSurface、eglContext等，调用makeCurrent关联我们后续的绘制行为。
2）根据设定的顶点着色器与片段着色器加载shader，获得shaderProgram
3）获取Camera数据，生成2D纹理。如果有多个摄像头，需要生成多个对应的纹理。
4）根据PNG切图，生成对应的2D纹理。
以上完成Render前的基础设定，然后我们调用drawFrame接口来实现具体的绘制。这里面也可以进行细分：
1）通过attachRenderTarget接口将Camera数据绑定到Graphicbuffer中。
2）计算顶点坐标，设定纹理坐标，这里面可能会包含一些图像旋转、翻转、镜像等操作。
3）更新2D纹理
4）调用glFinish将所有的OpenGL指令进行执行
5）最后调用detachRenderTarget，结束此帧画面的绘制
以上就是EVS应用中核心部分的解析了，关于图像绘制这块其实还有很多知识可以进一步挖掘，会涉及到大量的OpenGL ES知识，本篇博文不做过多深入，感兴趣的朋友可以自行了解。
最后放上一个流程图，帮助大家更好的理解：
CN 
EN