这篇文章上次修改于 1365 天前,可能其部分内容已经发生变化,如有疑问可询问作者。
title: 提取腾讯视频软字幕
date: 2021/03/13 23:30:00
updated: 2021/03/14 23:30:00
permalink: extract-tencent-video-subtitle/
toc: true
环境
- Kali-Linux-2019.4-vmware-amd64
- frida-server-14.2.13-android-arm64
- TencentVideo_V8.3.25.21854.apk
背景和目标
腾讯视频有一个台词海报的功能,该功能可以将一句或者多句台词生成海报,用户可以分享给他人
这个功能出现很久了,使用时稍加注意就可以发现这样的情形:点击台词背景缩略图会变化,也就是说台词有对应视频时间位置
本质上来说,它就是软字幕
腾讯视频拓展海外,平台是WETV,在WETV则直接能从视频接口拿到字幕链接
腾讯国内的视频即使有台词海边这个功能,但视频内容仍然是硬字幕的
不过不重要,关键是如果想要这个台词海报(软字幕)该怎么办呢
这就是本文的目标了
分析
首先找一个有台词海报的视频,大江大河2第1集(通常来说电视剧才有台词海报功能)
台词海报具体分享页面如下
既然可以点点点,那不得直接用hookEvent.js
安排
然而选择一句台词,并没有反应
啊这...不要紧,试试前面的台词海报按钮
有了!定位到的类是
com.tencent.qqlive.ona.player.plugin.operate.helper.VodSwMoreOperateCaptionIconHelper
然而这种情况也不知道如何继续追踪...
咋办呢?这个台词内容显然会放到android.widget.TextView
!
当然是objection
一把梭
android hooking watch class_method android.widget.TextView.setText "java.lang.CharSequence" --dump-args
显然!
然后得到一份调用栈
(agent) [135218] Called android.widget.TextView.setText(java.lang.CharSequence)
(agent) [135218] Backtrace:
android.widget.TextView.setText(Native Method)
com.tencent.qqlive.ona.share.caption.CaptionEditor.a(CaptionEditor.java:231)
com.tencent.qqlive.ona.share.caption.CaptionEditor.addData(CaptionEditor.java:173)
com.tencent.qqlive.ona.share.caption.CaptionEditActivity$2.onCaptionChoiceChange(CaptionEditActivity.java:172)
com.tencent.qqlive.ona.share.caption.CaptionListAdapter.onLoadFinish(CaptionListAdapter.java:106)
com.tencent.qqlive.ona.model.base.a$2$1.a(BaseModel.java:51)
com.tencent.qqlive.ona.model.base.a$2$1.onNotify(BaseModel.java:48)
com.tencent.qqlive.utils.v.a(ListenerMgr.java:85)
com.tencent.qqlive.ona.model.base.a$2.run(BaseModel.java:48)
android.os.Handler.handleCallback(Handler.java:873)
android.os.Handler.dispatchMessage(Handler.java:99)
android.os.Looper.loop(Looper.java:201)
android.app.ActivityThread.main(ActivityThread.java:6882)
java.lang.reflect.Method.invoke(Native Method)
com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:547)
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:873)
(agent) [135218] Arguments android.widget.TextView.setText(当初我把我姐)
关注
com.tencent.qqlive.ona.model.base.a$2.run
既然调用了run
那么肯定要初始化的,那么hook初始化的调用栈
(agent) [989534] Called com.tencent.qqlive.ona.model.base.a$2.a$2(com.tencent.qqlive.ona.model.base.a, com.tencent.qqlive.ona.model.base.a, int, boolean, boolean)
(agent) [989534] Backtrace:
com.tencent.qqlive.ona.model.base.a$2.<init>(Native Method)
com.tencent.qqlive.ona.model.base.a.sendMessageToUI(BaseModel.java:44)
com.tencent.qqlive.ona.model.base.a.sendMessageToUI(BaseModel.java:61)
com.tencent.qqlive.ona.model.base.d.a(BasePreGetNextPageModel.java:301)
com.tencent.qqlive.ona.model.base.d.a(BasePreGetNextPageModel.java:279)
com.tencent.qqlive.ona.circle.c.i.a(FriendsScreenShotModel.java:225)
com.tencent.qqlive.ona.model.base.d.onProtocolRequestFinish(BasePreGetNextPageModel.java:251)
com.tencent.qqlive.route.ProtocolManager.onNetWorkFinish(ProtocolManager.java:405)
com.tencent.qqlive.route.n.a(NetWorkTask.java:377)
com.tencent.qqlive.route.n.a(NetWorkTask.java:314)
com.tencent.qqlive.x.i$1.a(NetworkConfig.java:223)
com.tencent.qqlive.x.i$1.a(NetworkConfig.java:105)
com.tencent.qqlive.route.r.a(RouteConfig.java:234)
com.tencent.qqlive.route.r.a(RouteConfig.java:220)
com.tencent.qqlive.route.n.a(NetWorkTask.java:144)
com.tencent.qqlive.route.n.a(NetWorkTask.java:122)
com.tencent.qqlive.route.n.run(NetWorkTask.java:108)
com.tencent.qqlive.modules.vb.threadservice.a.d.run(VBExecuteRunnable.java:48)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
java.lang.Thread.run(Thread.java:764)
(agent) [989534] Arguments com.tencent.qqlive.ona.model.base.a$2.a$2(com.tencent.qqlive.ona.circle.c.i@b9fc8cc, com.tencent.qqlive.ona.circle.c.i@b9fc8cc, (none), true, true)
这个时候观察到一个FriendsScreenShotModel
反编译代码中有一个CircleGetScreenShotsResponse
类型
这样一看就有意思了
在com.tencent.qqlive.ona.protocol.jce.CircleGetScreenShotsResponse.readFrom
这里做更详细的hook
类com.qq.taf.jce.JceInputStream
的相关函数如下
private <K,V> java.util.Map<K, V> com.qq.taf.jce.JceInputStream.readMap(java.util.Map<K, V>,java.util.Map<K, V>,int,boolean)
private <T> T[] com.qq.taf.jce.JceInputStream.readArrayImpl(T,int,boolean)
private int com.qq.taf.jce.JceInputStream.peakHead(com.qq.taf.jce.JceInputStream$HeadData)
private void com.qq.taf.jce.JceInputStream.skip(int)
private void com.qq.taf.jce.JceInputStream.skipField()
private void com.qq.taf.jce.JceInputStream.skipField(byte)
public <K,V> java.util.HashMap<K, V> com.qq.taf.jce.JceInputStream.readMap(java.util.Map<K, V>,int,boolean)
public <T> T[] com.qq.taf.jce.JceInputStream.readArray(T[],int,boolean)
public <T> java.lang.Object com.qq.taf.jce.JceInputStream.read(T,int,boolean)
public <T> java.util.List<T> com.qq.taf.jce.JceInputStream.readArray(java.util.List<T>,int,boolean)
public boolean com.qq.taf.jce.JceInputStream.read(boolean,int,boolean)
public boolean com.qq.taf.jce.JceInputStream.skipToTag(int)
public boolean[] com.qq.taf.jce.JceInputStream.read(boolean[],int,boolean)
public byte com.qq.taf.jce.JceInputStream.read(byte,int,boolean)
public byte[] com.qq.taf.jce.JceInputStream.read(byte[],int,boolean)
public com.qq.taf.jce.JceStruct com.qq.taf.jce.JceInputStream.directRead(com.qq.taf.jce.JceStruct,int,boolean)
public com.qq.taf.jce.JceStruct com.qq.taf.jce.JceInputStream.read(com.qq.taf.jce.JceStruct,int,boolean)
public com.qq.taf.jce.JceStruct[] com.qq.taf.jce.JceInputStream.read(com.qq.taf.jce.JceStruct[],int,boolean)
public double com.qq.taf.jce.JceInputStream.read(double,int,boolean)
public double[] com.qq.taf.jce.JceInputStream.read(double[],int,boolean)
public float com.qq.taf.jce.JceInputStream.read(float,int,boolean)
public float[] com.qq.taf.jce.JceInputStream.read(float[],int,boolean)
public int com.qq.taf.jce.JceInputStream.read(int,int,boolean)
public int com.qq.taf.jce.JceInputStream.setServerEncoding(java.lang.String)
public int[] com.qq.taf.jce.JceInputStream.read(int[],int,boolean)
public java.lang.String com.qq.taf.jce.JceInputStream.read(java.lang.String,int,boolean)
public java.lang.String com.qq.taf.jce.JceInputStream.readByteString(java.lang.String,int,boolean)
public java.lang.String com.qq.taf.jce.JceInputStream.readString(int,boolean)
public java.lang.String[] com.qq.taf.jce.JceInputStream.read(java.lang.String[],int,boolean)
public java.nio.ByteBuffer com.qq.taf.jce.JceInputStream.getBs()
public java.util.List com.qq.taf.jce.JceInputStream.readList(int,boolean)
public java.util.Map<java.lang.String, java.lang.String> com.qq.taf.jce.JceInputStream.readStringMap(int,boolean)
public long com.qq.taf.jce.JceInputStream.read(long,int,boolean)
public long[] com.qq.taf.jce.JceInputStream.read(long[],int,boolean)
public short com.qq.taf.jce.JceInputStream.read(short,int,boolean)
public short[] com.qq.taf.jce.JceInputStream.read(short[],int,boolean)
public static int com.qq.taf.jce.JceInputStream.readHead(com.qq.taf.jce.JceInputStream$HeadData,java.nio.ByteBuffer)
public void com.qq.taf.jce.JceInputStream.readHead(com.qq.taf.jce.JceInputStream$HeadData)
public void com.qq.taf.jce.JceInputStream.skipToStructEnd()
public void com.qq.taf.jce.JceInputStream.warp(byte[])
public void com.qq.taf.jce.JceInputStream.wrap(byte[])
结合反编译代码,最终的操作都是落在了java.nio.ByteBuffer
那在readFrom
时直接那一份二进制内容出来吧!
尝试了很多(省略N多过程),然后发现这个点没有找到数据
还是要在最开始找到数据的最近的地方追踪
对com.tencent.qqlive.ona.share.caption.CaptionEditor
进行追踪
com.tencent.qqlive.ona.share.caption.CaptionEditor.a
com.tencent.qqlive.ona.share.caption.CaptionEditor.addData
可以发现在点击选择其他字幕的时候,会调用addData
(agent) [ouzhno7bvfi] Called com.tencent.qqlive.ona.share.caption.CaptionEditor.addData(int, com.tencent.qqlive.ona.protocol.jce.CaptionInfo)
(agent) [ouzhno7bvfi] Arguments com.tencent.qqlive.ona.share.caption.CaptionEditor.addData(1, "<instance: com.tencent.qqlive.ona.protocol.jce.CaptionInfo>")
然后找一个com.tencent.qqlive.ona.protocol.jce.CaptionInfo
实例看看
那现在应该继续追踪com.tencent.qqlive.ona.protocol.jce.CaptionInfo
了
调用栈如下
(agent) [o1dgw84u3vm] Called com.tencent.qqlive.ona.protocol.jce.CaptionInfo.CaptionInfo()
(agent) [o1dgw84u3vm] Backtrace:
com.tencent.qqlive.ona.protocol.jce.CaptionInfo.<init>(Native Method)
java.lang.Class.newInstance(Native Method)
com.qq.taf.jce.JceInputStream.read(JceInputStream.java:933)
com.qq.taf.jce.JceInputStream.read(JceInputStream.java:978)
com.qq.taf.jce.JceInputStream.readArrayImpl(JceInputStream.java:839)
com.qq.taf.jce.JceInputStream.readArray(JceInputStream.java:819)
com.qq.taf.jce.JceInputStream.read(JceInputStream.java:976)
com.tencent.qqlive.ona.protocol.jce.GetCaptionResponse.readFrom(GetCaptionResponse.java:54)
com.tencent.qqlive.route.ProtocolPackage.unPackageJceResponse(ProtocolPackage.java:156)
com.tencent.qqlive.route.n.a(NetWorkTask.java:312)
com.tencent.qqlive.x.i$1.a(NetworkConfig.java:223)
com.tencent.qqlive.x.i$1.a(NetworkConfig.java:105)
com.tencent.qqlive.route.r.a(RouteConfig.java:234)
com.tencent.qqlive.route.r.a(RouteConfig.java:220)
com.tencent.qqlive.route.n.a(NetWorkTask.java:144)
com.tencent.qqlive.route.n.a(NetWorkTask.java:122)
com.tencent.qqlive.route.n.run(NetWorkTask.java:108)
com.tencent.qqlive.modules.vb.threadservice.a.d.run(VBExecuteRunnable.java:48)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
java.lang.Thread.run(Thread.java:764)
(agent) [o1dgw84u3vm] Return Value: "(none)"
根据反编译代码可以看到,在readFrom
后就会得到字幕
编写hook脚本如下
function HookCaptionInfo(){
Java.perform(function(){
let CaptionInfo = Java.use("com.tencent.qqlive.ona.protocol.jce.CaptionInfo");
let ByteBuffer = Java.use("java.nio.ByteBuffer");
CaptionInfo.readFrom.overload("com.qq.taf.jce.JceInputStream").implementation = function(inputstream){
let field_bs = inputstream.class.getDeclaredField("bs");
field_bs.setAccessible(true);
let bs = inputstream.bs.value;
if (flag == false){
// get raw data from buffer
flag = true;
console.log(inputstream);
let position = bs.position();
let length = bs.capacity();
console.log('info', position, length);
// get a copy of bs
let copy = ByteBuffer.allocate(bs.capacity());
bs.position(0)
copy.put(bs);
bs.position(position)
copy.flip();
// transfer copy to buffer
let buffer = Java.array('byte', new Uint8Array(length));
copy.get(buffer);
send({"buffer":length}, new Uint8Array(buffer));
}
let ret = this.readFrom(inputstream);
return ret;
}
})
}
var flag = false
setImmediate(HookCaptionInfo)
然后编写python脚本接收数据,保存到本地之后是这个样子的(第37集)
但是如何解析成更加正常一点的结果呢
继续追踪就是com.qq.taf.jce.JceInputStream
了
这里面有很多操作,解析它自定义结构体的
尝试还原了一部分,但是发现工作量太大...暂时搁置
继续追踪请求来源,再往前看一个调用栈
也就是com.tencent.qqlive.route.ProtocolPackage.unPackageJceResponse
嗯哼,请求就是这里在发起了
可以看到有一个buildPostData
方法,追踪它
这样一来就可以确认com.tencent.qqlive.route.jce.RequestCommand.writeTo(com.qq.taf.jce.JceOutputStream)
就是需要关注的,依葫芦画瓢,把请求的内容拿下来
可以看到熟悉的参数,红线的参数就是视频的cid和vid了
其他部分含有大量的环境信息,已水印
- IMEI
- 手机型号
- APP安装时间
- APP版本
- session
这里不用过于关心这些参数怎么来,我的目的是寻找更加上层的位置
达成通过视频cid
+vid
直接获取返回
追踪com.tencent.qqlive.route.jce.RequestCommand.writeTo
(agent) [890vo80ecxr] Called com.tencent.qqlive.route.jce.RequestCommand.writeTo(com.qq.taf.jce.JceOutputStream)
(agent) [890vo80ecxr] Backtrace:
com.tencent.qqlive.route.jce.RequestCommand.writeTo(Native Method)
com.tencent.qqlive.route.ProtocolPackage.jceStructToUTF8Byte(ProtocolPackage.java:177)
com.tencent.qqlive.route.ProtocolPackage.buildPostData(ProtocolPackage.java:73)
com.tencent.qqlive.route.t.a(UnifiedProtocolUtils.java:47)
com.tencent.qqlive.route.n.f(NetWorkTask.java:194)
com.tencent.qqlive.route.n.run(NetWorkTask.java:97)
com.tencent.qqlive.modules.vb.threadservice.a.d.run(VBExecuteRunnable.java:48)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
java.lang.Thread.run(Thread.java:764)
(agent) [890vo80ecxr] Arguments com.tencent.qqlive.route.jce.RequestCommand.writeTo("<instance: com.qq.taf.jce.JceOutputStream>")
com.tencent.qqlive.route.t.a
方法如下
这里显然还不够顶层,仍然是在操作基础数据
com.tencent.qqlive.route.n.f
方法中有一个packageRequest
,一并追踪
可以发现packageRequest
有参数是com.qq.taf.jce.JceStruct
实现类是com.tencent.qqlive.ona.protocol.jce.WatchRecordUploadV1Request
还是JceStruct
,那么依然不够顶层
再往上看就是com.tencent.qqlive.route.n.run
了,那么关注一下它的初始化
得到一份调用栈
(agent) [oo4t2ej42c] Called com.tencent.qqlive.route.n.n(com.tencent.qqlive.route.jce.ServerInfo, int, int)
(agent) [oo4t2ej42c] Backtrace:
com.tencent.qqlive.route.n.<init>(Native Method)
com.tencent.qqlive.route.ProtocolManager.a(ProtocolManager.java:380)
com.tencent.qqlive.route.ProtocolManager.a(ProtocolManager.java:338)
com.tencent.qqlive.route.ProtocolManager.sendRequest(ProtocolManager.java:304)
com.tencent.qqlive.route.ProtocolManager.sendRequest(ProtocolManager.java:268)
com.tencent.qqlive.route.ProtocolManager.sendRequest(ProtocolManager.java:218)
com.tencent.qqlive.route.ProtocolManager.sendRequest(ProtocolManager.java:120)
com.tencent.qqlive.ona.share.caption.GetCaptionModel.loadData(GetCaptionModel.java:59)
com.tencent.qqlive.ona.share.caption.CaptionListAdapter.loadData(CaptionListAdapter.java:64)
com.tencent.qqlive.ona.share.caption.CaptionEditActivity.f(CaptionEditActivity.java:427)
com.tencent.qqlive.ona.share.caption.CaptionEditActivity.onCreate(CaptionEditActivity.java:121)
android.app.Activity.performCreate(Activity.java:7232)
android.app.Activity.performCreate(Activity.java:7221)
android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1272)
com.tencent.qqlive.ac.a.a$b.callActivityOnCreate(ActivityLifeCycleMonitor.java:104)
android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2971)
android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3126)
android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
android.app.ActivityThread$H.handleMessage(ActivityThread.java:1846)
android.os.Handler.dispatchMessage(Handler.java:106)
android.os.Looper.loop(Looper.java:201)
android.app.ActivityThread.main(ActivityThread.java:6882)
java.lang.reflect.Method.invoke(Native Method)
com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:547)
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:873)
(agent) [oo4t2ej42c] Arguments com.tencent.qqlive.route.n.n("<instance: com.tencent.qqlive.route.jce.ServerInfo>", 103, 3)
可以看到ProtocolManager
的sendRequest
多次重载,显然不需要关心它
看看com.tencent.qqlive.ona.share.caption.CaptionListAdapter.loadData
看来是合适的地方了,验证一下
很好!
要不要再往前一个栈看呢?我的答案是不需要了
因为可以看到com.tencent.qqlive.ona.share.caption.CaptionListAdapter
有onCreate
方法
显然这会涉及到界面了,那要主动构造调用就比较麻烦
现在仔细看看com.tencent.qqlive.ona.share.caption.CaptionListAdapter
可以看到初始化后就调用了loadData
方法
那么主动构造参数调用就比较简单了
vid和cid好说,有两个数字是怎么来的呢?
结合CaptionListAdapter
稍作分析就能知道
- 第一个是点击台词海报按钮时观看视频的位置(秒)
- 第二个是视频总时长(毫秒)
具体解析也不用自己动手了,直接通过GetCaptionModel
的属性去把结果取出来,然后转换成为需要的格式
...
在com.tencent.qqlive.route.k.a
进行overload可以得到解密后的结果
在com.tencent.qqlive.ona.protocol.jce.GetCaptionResponse
进行overload可以得到解析好的结果
...
本来这里有很多分析过程,但是没有记录,直接上代码了
下面的hook代码作用是截获点击台词海报
按钮时,服务器返回的数据
这里需要复制一份解密(?)后的数据,需要处理一下
flag
的作用是只取一次即可,因为一个CaptionInfo
对应单条台词,这里会多次进入
function HookCaptionInfo(){
Java.perform(function(){
let CaptionInfo = Java.use("com.tencent.qqlive.ona.protocol.jce.CaptionInfo");
let ByteBuffer = Java.use("java.nio.ByteBuffer");
CaptionInfo.readFrom.overload("com.qq.taf.jce.JceInputStream").implementation = function(inputstream){
let field_bs = inputstream.class.getDeclaredField("bs");
field_bs.setAccessible(true);
let bs = inputstream.bs.value;
if (flag == false){
// get raw data from buffer
flag = true;
console.log(inputstream);
let position = bs.position();
let length = bs.capacity();
console.log('info', position, length);
// get a copy of bs
let copy = ByteBuffer.allocate(bs.capacity());
bs.position(0)
copy.put(bs);
bs.position(position)
copy.flip();
// transfer copy to buffer
let buffer = Java.array('byte', new Uint8Array(length));
copy.get(buffer);
send({"buffer":length}, new Uint8Array(buffer));
}
let ret = this.readFrom(inputstream);
return ret;
}
})
}
var flag = false
setImmediate(HookCaptionInfo)
然后经过N久的分析、跳转、寻找、定位(太乱了以至于没有记录)...
终于找到了合适的hook点,可以达成主动调用,获取任意视频的台词海报内容(字幕,其实是腾讯自己OCR的,有一些错误)
不过不完全是主动调用,请求是主动调用,获取返回结果是通过GetCaptionResponse.readFrom
处拦截的
下面是代码,用到了三方库,需要使用frida-compile
进行编译,然后才能用于注入
const Koa = require('koa');
const Router = require('koa-router');
const server = new Koa();
const router = new Router();
router.get('/subtitle/:vid', (ctx, next) => {
ctx.body = GetSubtitle(ctx.params.vid, '');
ctx.type = 'application/json; charset=utf-8';
});
server.use(router.routes()).use(router.allowedMethods()).listen(16666);
function GetSubtitle(vid, cid) {
var resp = {"name": "", "subtitles": new Array()};
Java.perform(function () {
function CaptionListToSRT(CaptionList) {
for (let i = 0; i < CaptionList.size(); i++) {
let info = Java.cast(CaptionList.get(i), CaptionInfo);
let item = {
"number": info.captionId.value,
"time": info.captionTime.value,
"caption": info.caption.value,
"status": info.status.value,
"duration": info.captionDispearTime.value,
}
resp.subtitles.push(item);
}
}
let CaptionInfo = Java.use("com.tencent.qqlive.ona.protocol.jce.CaptionInfo");
let GetCaptionModel = Java.use("com.tencent.qqlive.ona.share.caption.GetCaptionModel");
let GetCaptionResponse = Java.use("com.tencent.qqlive.ona.protocol.jce.GetCaptionResponse");
GetCaptionResponse.readFrom.overload('com.qq.taf.jce.JceInputStream').implementation = function (a) {
let ret = this.readFrom(a);
resp.name = this.title.value;
CaptionListToSRT(this.captionList.value);
return ret
}
let model = GetCaptionModel.$new();
model.loadData("vid=" + vid, 0, 0, cid, vid);
Thread.sleep(0.8);
})
return resp;
}
编译建议配合frida-agent-example
食用
最终效果图
没有评论