众所周知JVM执行的是.class字节码,而Android的Dalvik或ART是不能直接运行.class文件,运行的是编译生成的dex文件,并且不同版本的Android在运行时对dex的编译策略也存在着变化。在Android4.4之前Android虚拟机是Dalvik,在Android2.2到Android4.4时期,其执行的时候采用的是JIT编译,当App运行时,每当遇到一个新类,JIT编译器就会对这个类进行即时编译,经过编译后的代码,会被优化成相当精简的原生型指令码,这样在下次执行到相同逻辑的时候,速度就会更快;在Android4.4的时候Google推出了新的虚拟机运行时环境ART,在Android7.0之前,其采用的是AOT编译,应用在安装的时候会启动 dex2oat,把 dex预编译成 ELF 文件,每次运行程序的时候不用重新编译。以上两种方式存在着缺陷很明显:JIT每次程序启动后都要重新编译,在运行时也会额外对性能消耗;AOT在apk安装时要消耗大量时间,且编译优化后的文件又会额外占用磁盘空间。所以在Android7.0时采用AOT/JIT 混合编译的策略:apk安装时dex文件不会被编译,app运行时dex文件会解释执行,热点函数被JIT编译后缓存并生成profile文件,在手机空闲或者充电时进行AOT编译。
本文将基于Android10的源码分析Apk安装过程中,对dex的编译处理;并给出一种基于热代码配置文件的安装方案。
dex2oat
dex编译优化成为运行时执行的oat的核心是dex2oat,所以首先介绍一下dex2oat。dex2oat是ART中把dex文件编译成oat的工具,位置在手机的/system/bin/dex2oat,源码在art/dex2oat/Dex2oat.cc。执行dex2oat编译时的编译选项中有compile-filter,从 Android O 开始,有四个官方支持的过滤器:
verify:只运行 DEX 代码验证。
quicken:运行 DEX 代码验证,并优化一些 DEX 指令,以获得更好的解译器性能。
speed:运行 DEX 代码验证,并对所有方法进行 AOT 编译。
speed-profile:运行 DEX 代码验证,并对配置文件中列出的方法进行 AOT 编译。
apk安装
接下来我们分析Android安装apk时的流程,apk安装可以分为四步:将APK的信息通过IO流的形式写入到PackageInstaller.Session中;调用PackageInstaller.Session的commit方法,将APK的信息交由PKMS处理;拷贝Apk文件到/data/app等目录;执行安装。
我们跳过前面的步骤,直接分析真正安装的代码。复制完Apk后调用HandlerParams,执行PackageManagerService中的安装方法。
//PackageManagerService.java
private void processPendingInstall(final InstallArgs args, final int currentStatus) {
if (args.mMultiPackageInstallParams != null) {
args.mMultiPackageInstallParams.tryProcessInstallRequest(args, currentStatus);
} else {
//普通安装
PackageInstalledInfo res = createPackageInstalledInfo(currentStatus);
processInstallRequestsAsync(res.returnCode == PackageManager.INSTALL_SUCCEEDED, Collections.singletonList(new InstallRequest(args, res)));
}
}
//processInstallRequestsAsync中会调用到installPackagesLI
private void installPackagesLI(List<InstallRequest> requests) {
//分析安装包并对其进行初始验证
final PrepareResult prepareResult = preparePackageLI(request.args, request.installResult);
final List<ScanResult> scanResults = scanPackageTracedLI(
prepareResult.packageToScan, prepareResult.parseFlags,
prepareResult.scanFlags, System.currentTimeMillis(),
request.args.user);
ReconcileRequest reconcileRequest = new ReconcileRequest(preparedScans, installArgs,
installResults,
prepareResults,
mSharedLibraries,
Collections.unmodifiableMap(mPackages), versionInfos,
lastStaticSharedLibSettings);
commitRequest = new CommitRequest(reconciledPackages, sUserManager.getUserIds());
commitPackagesLocked(commitRequest);
//执行完成Apk安装
executePostCommitSteps(commitRequest);
}
private void executePostCommitSteps(CommitRequest commitRequest) {
for (ReconciledPackage reconciledPkg : commitRequest.reconciledPackages.values()) {
//根据解析的包准备完数据进行安装
prepareAppDataAfterInstallLIF(pkg);
//如果需要替换安装,则需要清楚原有的APP数据
if (reconciledPkg.prepareResult.clearCodeCache) {
clearAppDataLIF(pkg, UserHandle.USER_ALL, FLAG_STORAGE_DE | FLAG_STORAGE_CE | FLAG_STORAGE_EXTERNAL | Installer.FLAG_CLEAR_CODE_CACHE_ONLY);
}
//如果需要更新,通知dex有更新
if (reconciledPkg.prepareResult.replace) {
mDexManager.notifyPackageUpdated(pkg.packageName, pkg.baseCodePath, pkg.splitCodePaths);
}
//为新的代码路径准备应用程序配置文件
mArtManagerService.prepareAppProfiles(pkg, resolveUserIds(reconciledPkg.installArgs.user.getIdentifier()),
final boolean performDexopt =
(!instantApp || Global.getInt(mContext.getContentResolver(),
Global.INSTANT_APP_DEXOPT_ENABLED, 0) != 0)
&& ((pkg.applicationInfo.flags & ApplicationInfo.FLAG_DEBUGGABLE) == 0);
if (performDexopt) {
DexoptOptions dexoptOptions = new DexoptOptions(packageName,
REASON_INSTALL,
DexoptOptions.DEXOPT_BOOT_COMPLETE
| DexoptOptions.DEXOPT_INSTALL_WITH_DEX_METADATA_FILE);
//执行dex优化
mPackageDexOptimizer.performDexOpt(pkg, null /* instructionSets */, getOrCreateCompilerPackageStats(pkg), mDexManager.getPackageUseInfoOrDefault(packageName),
dexoptOptions);
}
BackgroundDexOptService.notifyPackageChanged(packageName);
}
}
}
使用profile安装apk
Android 7.0 引入了 Profile Guided Optimization(PGO),允许 Android 运行时收集热门代码的配置,并集中编译提升应用性能,可以减少完全编译的应用耗时和磁盘空间占用,但这种模式需要app运行一段时间后才能感受到性能的优化。在Improving app performance with ART optimizing profiles in the cloud 文章中,Google通过收集众多用户的热门代码,构建核心聚合代码配置文件,Google Play就会将代码配置和Apk一同安装,在安装时实现有效的配置来引导优化,从而改善冷启动时间以及稳定性能状态。
Google Play中的配置文件.dm的内容和热带码的primary.prof中的内容是一致的。使用profman --profile-file=/data/misc/profiles/cur/0/package name/primary.prof --dump-only
可以查看prof中的内容:
也可以使用cmd package dump-profiles package name
,dump为一个txt文件到/data/misc/profman。
Google Play安装Apk时实际上使用的是pm install命令,我们可以用命令模拟一遍这种带.dm的安装过程:
1.app安装运行一段时间之后,得到/data/misc/profiles/cur/0/package name/primary.prof,将primary.prof压缩为zip改为dm格式;
2.执行安装命令:
pm install-create
pm install-write SESSION_ID AppName.apk /data/local/tmp/AppName.apk
//比正常安装多执行下面一行
pm install-write SESSION_ID AppName.dm /data/local/tmp/AppName.dm
pm install-commit SESSION_ID
下面通过源码分析安装Apk时profile的处理,上一节中已经分析到了PackageMangerService的executePostCommitSteps方法中为新的代码路径准备应用程序配置文件,下面进一步分析准备配置文件的代码。
//PackageManagerService.java
// Prepare the application profiles for the new code paths.
// This needs to be done before invoking dexopt so that any install-time profile
// can be used for optimizations.
mArtManagerService.prepareAppProfiles(pkg,resolveUserIds(reconciledPkg.installArgs.user.getIdentifier()),
/* updateReferenceProfileContent= */ true);
//ArtManagerService.jav
//给app准备profile,如果有dex metadata拷贝到reference profile
public void prepareAppProfiles(PackageParser.Package pkg, @UserIdInt int user,
boolean updateReferenceProfileContent) {
ArrayMap<String, String> codePathsProfileNames = getPackageProfileNames(pkg);
for (int i = codePathsProfileNames.size() - 1; i >= 0; i--) {
String codePath = codePathsProfileNames.keyAt(i);
String profileName = codePathsProfileNames.valueAt(i);
String dexMetadataPath = null;
if (updateReferenceProfileContent) {
File dexMetadata = DexMetadataHelper.findDexMetadataForFile(new File(codePath));
dexMetadataPath = dexMetadata == null ? null : dexMetadata.getAbsolutePath();
}
synchronized (mInstaller) {
boolean result = mInstaller.prepareAppProfile(pkg.packageName, user, appId,
profileName, codePath, dexMetadataPath);
}
}
//Installer.java
public boolean prepareAppProfile(String pkg, @UserIdInt int userId, @AppIdInt int appId,
String profileName, String codePath, String dexMetadataPath) throws InstallerException {
return mInstalld.prepareAppProfile(pkg, userId, appId, profileName, codePath,dexMetadataPath);
}
//dexopt.cpp
bool prepare_app_profile(const std::string& package_name,
userid_t user_id,
appid_t app_id,
const std::string& profile_name,
const std::string& code_path,
const std::unique_ptr<std::string>& dex_metadata) {
// 将dex metdata合并到reference profile.
unique_fd ref_profile_fd = open_reference_profile(uid, package_name, profile_name,
/*read_write*/ true, /*is_secondary_dex*/ false);
unique_fd dex_metadata_fd(TEMP_FAILURE_RETRY(
open(dex_metadata->c_str(), O_RDONLY | O_NOFOLLOW)));
unique_fd apk_fd(TEMP_FAILURE_RETRY(open(code_path.c_str(), O_RDONLY | O_NOFOLLOW)));
if (apk_fd < 0) {
PLOG(ERROR) << "Could not open code path " << code_path;
return false;
}
RunProfman args;
args.SetupCopyAndUpdate(std::move(dex_metadata_fd),
std::move(ref_profile_fd),
std::move(apk_fd),
code_path);
// The copy and update takes ownership over the fds.
args.Exec();
}
return true;
}
分析完配置文件准备的执行过程,回到PackageManagerService中的安装流程,接下来PackageDexOptimizer调用performDexOpt执行编译优化dex。
//PackageDexOptimizer.java
private int performDexOptLI(PackageParser.Package pkg, String[] targetInstructionSets, CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo, DexoptOptions options) {
String dexMetadataPath = null;
if (options.isDexoptInstallWithDexMetadata()) {
//寻找是否有dex matedata文件
File dexMetadataFile = DexMetadataHelper.findDexMetadataForFile(new File(path));
dexMetadataPath = dexMetadataFile == null ? null : dexMetadataFile.getAbsolutePath();
}
//dexOptPath方法中调用Installer的dexopt
int newResult = dexOptPath(pkg, path, dexCodeIsa, compilerFilter,
profileUpdated, classLoaderContexts[i], dexoptFlags, sharedGid,
packageStats, options.isDowngrade(), profileName, dexMetadataPath,
options.getCompilationReason());
if ((result != DEX_OPT_FAILED) && (newResult != DEX_OPT_SKIPPED)) {
result = newResult;
}
}
private int dexOptPath(PackageParser.Package pkg, String path, String isa,
String compilerFilter, boolean profileUpdated, String classLoaderContext,
int dexoptFlags, int uid, CompilerStats.PackageStats packageStats, boolean downgrade,
String profileName, String dexMetadataPath, int compilationReason) {
//调用DexFile.getDexOptNeeded判断是否需要dex2oat编译,没有oat、oat文件不能匹配boot image、oat文件不能匹配compiler filter等情况
int dexoptNeeded = getDexoptNeeded(path, isa, compilerFilter, classLoaderContext, profileUpdated, downgrade);
//Installer中dexopt调用dexopt.cpp的dexopt
mInstaller.dexopt(path, uid, pkg.packageName, isa, dexoptNeeded, oatDir, dexoptFlags,
compilerFilter, pkg.volumeUuid, classLoaderContext, pkg.applicationInfo.seInfo,
false /* downgrade*/, pkg.applicationInfo.targetSdkVersion,
profileName, dexMetadataPath, getAugmentedReasonName(compilationReason, dexMetadataPath != null));
}
//dexopt.cpp
int dexopt(const char* dex_path, uid_t uid, const char* pkgname, const char* instruction_set,
int dexopt_needed, const char* oat_dir, int dexopt_flags, const char* compiler_filter,
const char* volume_uuid, const char* class_loader_context, const char* se_info,
bool downgrade, int target_sdk_version, const char* profile_name,
const char* dex_metadata_path, const char* compilation_reason, std::string* error_msg) {
//打开dex输入文件
unique_fd input_fd(open(dex_path, O_RDONLY, 0));
std::vector<unique_fd> context_input_fds;
if (open_dex_paths(context_dex_paths, &context_input_fds, error_msg) != 0) {
LOG(ERROR) << *error_msg;
return -1;
}
//创建oat输出文件
char out_oat_path[PKG_PATH_MAX];
//打开reference profile
Dex2oatFileWrapper reference_profile_fd = maybe_open_reference_profile(
pkgname, dex_path, profile_name, profile_guided, is_public, uid, is_secondary_dex);
//打开dex matedata文件
unique_fd dex_metadata_fd;
if (dex_metadata_path != nullptr) {
dex_metadata_fd.reset(TEMP_FAILURE_RETRY(open(dex_metadata_path, O_RDONLY | O_NOFOLLOW)));
}
//拼装命令参数执行dex2oat
RunDex2Oat runner(input_fd.get(),
out_oat_fd.get(),
in_vdex_fd.get(),
out_vdex_fd.get(),
image_fd.get(),
dex_path,
out_oat_path,
swap_fd.get(),
instruction_set,
compiler_filter,
debuggable,
boot_complete,
background_job_compile,
reference_profile_fd.get(),
class_loader_context,
join_fds(context_input_fds),
target_sdk_version,
enable_hidden_api_checks,
generate_compact_dex,
dex_metadata_fd.get(),
compilation_reason);
runner.Exec(DexoptReturnCodes::kDex2oatExec);
}