我在之前的三篇文章中分别介绍了 ART Runtime 的背景知识,dex2oat 以及 compiler。这次,我们详细剖析一下另外一个之前没有很注意的模块,就是 patchoat

〇、 为什么 ART 需要 Relocate

在详细解释 patchoat 的代码前,我们先要回答一个问题。

在 ART 初始化 ImageSpace 的时候,会做 Relocate 动作,将 system/framework/arm/boot.oat boot.art 拷贝到 dalvik-cache 里。 而这个动作是默认开启的。 我想问下为何需要做这个 Relocate 动作?

因为需要随机化内存地址 (address space layout randomization)。

ART runtime 的 image space 文件 boot.art 以及通过 AOT 编译后的 framework API 文件 boot.oat 的默认加载地址是 0x70000000。这样会造成每个 Android 设备的系统的加载地址都是一样的,会引发一些安全问题(攻击者可以猜测内存结构)。为了使每台机器 的加载地址不同,所以在初始化 ImageSpace 的时候调用 Relocate 函数。Relocate 会随 机选择一个 offset,对 ImageSpace 和 oat 进行 patch,即调用 patchoat 命令

Relocate 方法只会在第一次启动的时候调用,如果 dalvik-cache 里已经存在,则不需要 Relocate。详见代码:

if (has_cache && ChecksumsMatch(system_filename.c_str(), cache_filename.c_str())) {
          // We already have a relocated version
          image_filename = &cache_filename;
          relocated_version_used = true;
        } else {
          // We cannot have a relocated version, Relocate the system one and use it.

          std::string reason;
          bool success;

          // Check whether we are allowed to relocate.
          if (!can_compile) {
            reason = "Image dex2oat disabled by -Xnoimage-dex2oat.";
            success = false;
          } else if (!ImageCreationAllowed(is_global_cache, &reason)) {
            // Whether we can write to the cache.
            success = false;
          } else {
            // Try to relocate.
            success = RelocateImage(image_location, cache_filename.c_str(), image_isa, &reason);
          }
          // ...
        }

http://androidxref.com/6.0.0_r1/xref/art/runtime/gc/space/image_space.cc#506

Relocate 方法会随机选择 offset,详见 ChooseRelocationOffsetDelta 方法:

static int32_t ChooseRelocationOffsetDelta(int32_t min_delta, int32_t max_delta) {
  CHECK_ALIGNED(min_delta, kPageSize);
  CHECK_ALIGNED(max_delta, kPageSize);
  CHECK_LT(min_delta, max_delta);

  std::default_random_engine generator;
  generator.seed(NanoTime() * getpid());
  std::uniform_int_distribution<int32_t> distribution(min_delta, max_delta);
  int32_t r = distribution(generator);
  if (r % 2 == 0) {
    r = RoundUp(r, kPageSize);
  } else {
    r = RoundDown(r, kPageSize);
  }
  CHECK_LE(min_delta, r);
  CHECK_GE(max_delta, r);
  CHECK_ALIGNED(r, kPageSize);
  return r;
}

http://androidxref.com/6.0.0_r1/xref/art/runtime/gc/space/image_space.cc#56

一、调用

之前我们详细讲了 Relocate 函数会调用 RelocateImage,而它便是负责执行 patchoat 的函数,详细代码如下:

// Relocate the image at image_location to dest_filename and relocate it by a random amount.
static bool RelocateImage(const char* image_location, const char* dest_filename,
                               InstructionSet isa, std::string* error_msg) {
  // We should clean up so we are more likely to have room for the image.
  if (Runtime::Current()->IsZygote()) {
    LOG(INFO) << "Pruning dalvik-cache since we are relocating an image and will need to recompile";
    PruneDalvikCache(isa);
  }

  std::string patchoat(Runtime::Current()->GetPatchoatExecutable());

  std::string input_image_location_arg("--input-image-location=");
  input_image_location_arg += image_location;

  std::string output_image_filename_arg("--output-image-file=");
  output_image_filename_arg += dest_filename;

  std::string input_oat_location_arg("--input-oat-location=");
  input_oat_location_arg += ImageHeader::GetOatLocationFromImageLocation(image_location);

  std::string output_oat_filename_arg("--output-oat-file=");
  output_oat_filename_arg += ImageHeader::GetOatLocationFromImageLocation(dest_filename);

  std::string instruction_set_arg("--instruction-set=");
  instruction_set_arg += GetInstructionSetString(isa);

  std::string base_offset_arg("--base-offset-delta=");
  StringAppendF(&base_offset_arg, "%d", ChooseRelocationOffsetDelta(ART_BASE_ADDRESS_MIN_DELTA,
                                                                    ART_BASE_ADDRESS_MAX_DELTA));
  // ...
  return Exec(argv, error_msg);
}

这个函数主要作用是设置 patchoat 的一些参数,比如:

  • --input-image-location=: 需要 patch 的 boot.art 的位置
  • --output-image-file=: patch 过的 boot.art 的 output 的位置
  • --input-oat-location=: 需要 patch 的 boot.oat 的位置
  • --output-oat-file=: patch 后 boot.oat 的 output 位置
  • --instruction-set=: 很简单,可以是 arm, arm64, x86, x86_64, mips, mips64, etc.
  • --base-offset-delta=: 这里会调用 ChooseRelocationOffsetDelta 函数随机选择一个 offset,patchoat 就会根据这个 offset 来修改 boot.artboot.oat 中的相关信息。

最后调用 Exec() 函数来启动 patchoat. Exec() 函数很简单,是一个 fork(), execv()waitpid() 的一个 wrapper,有兴趣的可以读一下,写的非常完善。

二、patchoat

patchoat 是一个独立的可执行文件,他在 Android 系统的位置是:/system/bin/patchoat。当然,我们也可以从他的 Android.mk makefile 中看到:

$(eval $(call build-art-executable,patchoat,$(PATCHOAT_SRC_FILES),libcutils,art/compiler,target,ndebug,$(patchoat_arch)))

PatchOat::patchoat 函数开始是一些参数解析的代码,主要调用在这里:

  if (have_image_files && have_oat_files) {
    TimingLogger::ScopedTiming pt("patch image and oat", &timings);
    ret = PatchOat::Patch(input_oat.get(), input_image_location, base_delta,
                          output_oat.get(), output_image.get(), isa, &timings,
                          output_oat_fd >= 0,  // was it opened from FD?
                          new_oat_out);
    // The order here doesn't matter. If the first one is successfully saved and the second one
    // erased, ImageSpace will still detect a problem and not use the files.
    ret = ret && FinishFile(output_image.get(), ret);
    ret = ret && FinishFile(output_oat.get(), ret);
  } else if (have_oat_files) {
    TimingLogger::ScopedTiming pt("patch oat", &timings);
    ret = PatchOat::Patch(input_oat.get(), base_delta, output_oat.get(), &timings,
                          output_oat_fd >= 0,  // was it opened from FD?
                          new_oat_out);
    ret = ret && FinishFile(output_oat.get(), ret);
  } else if (have_image_files) {
    TimingLogger::ScopedTiming pt("patch image", &timings);
    ret = PatchOat::Patch(input_image_location, base_delta, output_image.get(), isa, &timings);
    ret = ret && FinishFile(output_image.get(), ret);
  } else {
    CHECK(false);
    ret = true;
  }

http://androidxref.com/6.0.0_r1/xref/art/patchoat/patchoat.cc#1308

此处分为三种情况:

  1. have_image_files && have_oat_files
  2. have_oat_files
  3. have_image_files

这里接着上面的 RelocateImage 函数调用,属于第一种情况。所以调用相应的 PatchOat::Patch 函数。

关键的逻辑我给单独拿出来了:

  if (is_oat_pic >= ERROR_FIRST) {
    // Error logged by IsOatPic
    return false;
  } else if (is_oat_pic == PIC) {
    // Do not need to do ELF-file patching. Create a symlink and skip the ELF patching.
    if (!ReplaceOatFileWithSymlink(input_oat->GetPath(),
                                   output_oat->GetPath(),
                                   output_oat_opened_from_fd,
                                   new_oat_out)) {
      // Errors already logged by above call.
      return false;
    }
    // Don't patch the OAT, since we just symlinked it. Image still needs patching.
    skip_patching_oat = true;
  } else {
    CHECK(is_oat_pic == NOT_PIC);
  }

  PatchOat p(isa, elf.release(), image.release(), ispc->GetLiveBitmap(), ispc->GetMemMap(),
             delta, timings);
  t.NewTiming("Patching files");
  if (!skip_patching_oat && !p.PatchElf()) {
    LOG(ERROR) << "Failed to patch oat file " << input_oat->GetPath();
    return false;
  }
  if (!p.PatchImage()) {
    LOG(ERROR) << "Failed to patch image file " << input_image->GetPath();
    return false;
  }

  t.NewTiming("Writing files");
  if (!skip_patching_oat && !p.WriteElf(output_oat)) {
    LOG(ERROR) << "Failed to write oat file " << input_oat->GetPath();
    return false;
  }
  if (!p.WriteImage(output_image)) {
    LOG(ERROR) << "Failed to write image file " << input_image->GetPath();
    return false;
  }

总结一下代码的流程分为一下几个步骤

  1. 首先判断是否为 PIC (Position Independent Code),这里的 PIC 是指 boot.oat 这个 ELF 文件是否为 PIC。从 dex2oat 的代码剖析中,我们已经了解到所有的 oat 文件默认都不是 PIC 的。这里应该为以后考虑,未来如果可以生成 PIC 的 boot.oat, 这里不需要做额外的事情,只需要 symlink 到输出地址。
  2. 创建 PatchOat 类,这个类会在后面调用。
  3. p.PatchElf: 虽然这里叫 PatchElf,但是,里面的代码其实是对 oat 进行 patch,包括 Oat Header 和 text section 的patch。
  4. p.PatchImage: 很显然,这里需要修改一些 ImageSpace 中的内容,也就是 patch boot.art 文件
  5. p.WriteElf(), p.WriteImage: 最后,把修改的 boot.oatboot.art 写入文件中

我们重点看一下 PatchElf 和 PatchImage 的代码。

三、PatchElf

我们先来看一下 PatchElf: http://androidxref.com/6.0.0_r1/xref/art/patchoat/patchoat.cc#717

// 为了方便阅读,部分代码已删除
template <typename ElfFileImpl>
bool PatchOat::PatchElf(ElfFileImpl* oat_file) {
  oat_file->ApplyOatPatchesTo(".text", delta_);

  PatchOatHeader<ElfFileImpl>(oat_file);

  for (unsigned int i = 0; i < oat_file->GetProgramHeaderNum(); ++i) {
    auto hdr = oat_file->GetProgramHeader(i);
    if (hdr->p_type == PT_LOAD && hdr->p_vaddr == 0u) {
      need_boot_oat_fixup = false;
      break;
    }
  }
  if (!need_boot_oat_fixup) {
    // This is an app oat file that can be loaded at an arbitrary address in memory.
    // Boot image references were patched above and there's nothing else to do.
    return true;
  }

  // This is a boot oat file that's loaded at a particular address and we need
  // to patch all absolute addresses, starting with ELF program headers.

  // Fixup Phdr's
  oat_file->FixupProgramHeaders(delta_);

  // Fixup Shdr's
  oat_file->FixupSectionHeaders(delta_);

  oat_file->FixupDynamic(delta_);

  // Fixup dynsym
  oat_file->FixupSymbols(delta_, true);

  // Fixup symtab
  oat_file->FixupSymbols(delta_, false);

  oat_file->FixupDebugSections(delta_);
  1. ApplyOatPatchesTo .text,这个最重要,我们要单独拿出来解释。
  2. PatchOatHeader, 因为修改了 oat 在内存中的加载地址,所以我们也要相应的修改相关的 offset。
  3. Fixup Elf Headers, Section Headers, Dynamics, Elf Symbols, Debug Sections,同理,也要修改 ELF 相关的 header 和 symbol 的 offsets。

ApplyOatPatchesTo

ApplyOatPatchesTo 函数首先会在 ELF 找到一个特殊的 section,名字叫 target_section_name + ".oat_patches",如果是要 patch .text section 的话,我们要找到的特殊的 section 名字就叫 .text.oat_patches。找到后,就会调用 ApplyOatPatches

template <typename ElfTypes>
bool ElfFileImpl<ElfTypes>::ApplyOatPatchesTo(
    const char* target_section_name, Elf_Addr delta) {
  auto target_section = FindSectionByName(target_section_name);
  if (target_section == nullptr) {
    return true;
  }
  std::string patches_name = target_section_name + std::string(".oat_patches");
  auto patches_section = FindSectionByName(patches_name.c_str());
  if (patches_section == nullptr) {
    LOG(ERROR) << patches_name << " section not found.";
    return false;
  }
  if (patches_section->sh_type != SHT_OAT_PATCH) {
    LOG(ERROR) << "Unexpected type of " << patches_name;
    return false;
  }
  ApplyOatPatches(
      Begin() + patches_section->sh_offset,
      Begin() + patches_section->sh_offset + patches_section->sh_size,
      delta,
      Begin() + target_section->sh_offset,
      Begin() + target_section->sh_offset + target_section->sh_size);
  return true;
}

ApplyOatPatches

ApplyOatPatches 函数只做一件事,就是在 .text section 需要 patch 的地方加一个 delta offset。这时候我们就有个问题,这些 patch 的地址是如何生成的呢,哪些地址需要 patch,是由什么决定的呢?

// Apply LEB128 encoded patches to given section.
template <typename ElfTypes>
void ElfFileImpl<ElfTypes>::ApplyOatPatches(
    const uint8_t* patches, const uint8_t* patches_end, Elf_Addr delta,
    uint8_t* to_patch, const uint8_t* to_patch_end) {
  typedef __attribute__((__aligned__(1))) Elf_Addr UnalignedAddress;
  while (patches < patches_end) {
    to_patch += DecodeUnsignedLeb128(&patches);
    DCHECK_LE(patches, patches_end) << "Unexpected end of patch list.";
    DCHECK_LT(to_patch, to_patch_end) << "Patch past the end of section.";
    *reinterpret_cast<UnalignedAddress*>(to_patch) += delta;
  }
}

.oat_patches

在 ART Compiler 那片文章中我们详细介绍了从 bytecode 到 MIR 再到 LIR 的过程。其中 MIR->LIR 中由有一个函数叫 AssembleLIR, 其中有一步叫 InstallLiteralPools()。 Literal pool 类似一个字典,有每一个函数,变量的信息,Wikipedia 解释如下:

In computer science, and specifically in compiler and assembler design, a literal pool is a lookup table used to hold literals during assembly and execution.

patches_

在 InstallLiteralPools() 中我们就能看到,代码把需要 patch 的 literal 都给放入 patches_ 的一个 vector 里。后面在生成 oat 文件的时候,就会把一个个的 patch 地址给放入 .oat_patches section 里。

void Mir2Lir::InstallLiteralPools() {
// ...
  data_lir = code_literal_list_;
  while (data_lir != nullptr) {
    uint32_t target_method_idx = data_lir->operands[0];
    const DexFile* target_dex_file = UnwrapPointer<DexFile>(data_lir->operands[1]);
    patches_.push_back(LinkerPatch::CodePatch(code_buffer_.size(),
                                              target_dex_file, target_method_idx));
    PushUnpatchedReference(&code_buffer_);
    data_lir = NEXT_LIR(data_lir);
  }
// ...
}

http://androidxref.com/6.0.0_r1/xref/art/compiler/dex/quick/codegen_util.cc#489

每个对应的框架,也有自己的 InstallLiteralPools,这个函数会把 instruction 的 patch 放入 patches_ 中。

void ArmMir2Lir::InstallLiteralPools() {
  patches_.reserve(call_method_insns_.size() + dex_cache_access_insns_.size());

  // PC-relative calls to methods.
  for (LIR* p : call_method_insns_) {
    DCHECK_EQ(p->opcode, kThumb2Bl);
    uint32_t target_method_idx = p->operands[1];
    const DexFile* target_dex_file = UnwrapPointer<DexFile>(p->operands[2]);
    patches_.push_back(LinkerPatch::RelativeCodePatch(p->offset,
                                                      target_dex_file, target_method_idx));
  }
}

http://androidxref.com/6.0.0_r1/xref/art/compiler/dex/quick/arm/target_arm.cc#905

其中 call_method_insns_ 包括下面的一些情况(kDirect, kStatic 这类方法,调用了 CallWithLinkerFixup 函数):

LIR* ArmMir2Lir::GenCallInsn(const MirMethodLoweringInfo& method_info) {
  LIR* call_insn;
  if (method_info.FastPath() && ArmUseRelativeCall(cu_, method_info.GetTargetMethod()) &&
      (method_info.GetSharpType() == kDirect || method_info.GetSharpType() == kStatic) &&
      method_info.DirectCode() == static_cast<uintptr_t>(-1)) {
    call_insn = CallWithLinkerFixup(method_info.GetTargetMethod(), method_info.GetSharpType());
  } else {
    call_insn = OpReg(kOpBlx, TargetPtrReg(kInvokeTgt));
  }
  return call_insn;
}

http://androidxref.com/6.0.0_r1/xref/art/compiler/dex/quick/arm/call_arm.cc#749

Write

在写入 oat 的时候,Writer 会创建一个 .text.oat_patches 然后写入所有的 patches.

  std::unique_ptr<RawSection> text_oat_patches(new RawSection(
      ".text.oat_patches", SHT_OAT_PATCH));
  if (compiler_driver_->GetCompilerOptions().GetIncludePatchInformation()) {
    // Note that ElfWriter::Fixup will be called regardless and therefore
    // we need to include oat_patches for debug sections unconditionally.
    EncodeOatPatches(oat_writer->GetAbsolutePatchLocations(),
                     text_oat_patches->GetBuffer());
    builder->RegisterSection(text_oat_patches.get());
  }

http://androidxref.com/6.0.0_r1/xref/art/compiler/elf_writer_quick.cc#239

至此,对于 PatchElf 的分析结束了,patch 完编译后的 boot.oat,我们还需要对 boot.art 里面的相关信息进行 patch,这时候就开始调用 PatchImage 了。

四、PatchImage

看完了 PatchElf 函数,我们来研究一下 PatchImage。

bool PatchOat::PatchImage() {
  ImageHeader* image_header = reinterpret_cast<ImageHeader*>(image_->Begin());
  CHECK_GT(image_->Size(), sizeof(ImageHeader));
  // These are the roots from the original file.
  auto* img_roots = image_header->GetImageRoots();
  image_header->RelocateImage(delta_);

  PatchArtFields(image_header);
  PatchArtMethods(image_header);
  PatchInternedStrings(image_header);
  // Patch dex file int/long arrays which point to ArtFields.
  PatchDexFileArrays(img_roots);

  VisitObject(img_roots);
  if (!image_header->IsValid()) {
    LOG(ERROR) << "reloction renders image header invalid";
    return false;
  }

  {
    TimingLogger::ScopedTiming t("Walk Bitmap", timings_);
    // Walk the bitmap.
    WriterMutexLock mu(Thread::Current(), *Locks::heap_bitmap_lock_);
    bitmap_->Walk(PatchOat::BitmapCallback, this);
  }
  return true;
}

四个重要的函数:

  • PatchArtFields: patch ArtFileds section 中的 objects,地址加一个 delta
  • PatchArtMethods: patch ArtMethods section 中的 objects,地址加一个 delta
  • PatchInternedStrings: patch InternedStrings section 中的 objects,地址加一个 delta
  • PatchDexFileArrays: Patch dex file int/long arrays which point to ArtFields.

我们以 PatchArtMethods 为例看一下代码,其他的代码都类似。

void PatchOat::PatchArtMethods(const ImageHeader* image_header) {
  const auto& section = image_header->GetMethodsSection();
  const size_t pointer_size = InstructionSetPointerSize(isa_);
  size_t method_size = ArtMethod::ObjectSize(pointer_size);
  for (size_t pos = 0; pos < section.Size(); pos += method_size) {
    auto* src = reinterpret_cast<ArtMethod*>(heap_->Begin() + section.Offset() + pos);
    auto* dest = RelocatedCopyOf(src);
    FixupMethod(src, dest);
  }
}

http://androidxref.com/6.0.0_r1/xref/art/patchoat/patchoat.cc#429

代码循环遍历了 MethodsSection 中所有的 ArtMethod pointer,这些 pointer 是指向 boot.oat 中已经编译好的 Method native code。通过调用 FixupMethod 方法,把所有的 pointer 都加一个 delta offset。

void PatchOat::FixupMethod(ArtMethod* object, ArtMethod* copy) {
  const size_t pointer_size = InstructionSetPointerSize(isa_);
  copy->CopyFrom(object, pointer_size);
  // Just update the entry points if it looks like we should.
  // TODO: sanity check all the pointers' values
  copy->SetDeclaringClass(RelocatedAddressOfPointer(object->GetDeclaringClass()));
  copy->SetDexCacheResolvedMethods(RelocatedAddressOfPointer(object->GetDexCacheResolvedMethods()));
  copy->SetDexCacheResolvedTypes(RelocatedAddressOfPointer(object->GetDexCacheResolvedTypes()));
  copy->SetEntryPointFromQuickCompiledCodePtrSize(RelocatedAddressOfPointer(
      object->GetEntryPointFromQuickCompiledCodePtrSize(pointer_size)), pointer_size);
  copy->SetEntryPointFromInterpreterPtrSize(RelocatedAddressOfPointer(
      object->GetEntryPointFromInterpreterPtrSize(pointer_size)), pointer_size);
  copy->SetEntryPointFromJniPtrSize(RelocatedAddressOfPointer(
      object->GetEntryPointFromJniPtrSize(pointer_size)), pointer_size);
}

http://androidxref.com/6.0.0_r1/xref/art/patchoat/patchoat.cc#642

这里首先得到之前的 DeclaringClass, CacheResolvedMethods, DexCacheResolvedMethods, EntryPointFromInterpreterPtrSize, 等等的位置,然后调用 RelocatedAddressOfPointer 方法加一个 delta offset, 然后再把新的地址设置回去。

五、总结

patchoat 的整个流程其实很容易理解,为了移动 ART Runtime,需要分别对 boot.oat 中已经编译好的绝对地址进行 patch, 和 boot.art 中对应的 objects 存储的信息进行 patch。最后目标是达到移动之后的 ART Runtime 能够正常的运行。为了做到内存地址的随机,这样大费周折的移动整个 ART Runtime 真是不容易。当然,更好的做法应该是和 ELF 中的 PIC 一样,通过 GOT 来动态的查找地址,这样就可以避免固定的地址。当然,这里面还有性能和其他方面的限制,估计之后 Google 会慢慢完善这部份,让这部份变得更优雅一些。