avfoundation - iOS: Audio and Video tap from Airplay -

i have made video player analyzing realtime audio , video tracks video playing. videos stored on ios device (in apps documents directory).

this works fine. use mtaudioprocessingtap in order audio samples , fft, , analyzing video copy'ing pixel buffers played cmtime (the avplayer currenttime property). said, works fine.

but want support airplay. airplay not difficult, taps stop working airplay toggled , video playing on atv. somehow, mtaudioprocessingtap won't process , pixelbuffers empty... can't data.

is there way data ?

in order pixel buffers, fire event every few milli-sec , retrieving player's currenttime. then:

cvpixelbufferref imagebuffer = [videooutput copypixelbufferforitemtime:time itemtimefordisplay:nil]; cvpixelbufferlockbaseaddress(imagebuffer,0);  uint8_t *tempaddress = (uint8_t *) cvpixelbuffergetbaseaddress(imagebuffer);  size_t bytesperrow = cvpixelbuffergetbytesperrow(imagebuffer); size_t height = cvpixelbuffergetheight(imagebuffer);  cvpixelbufferunlockbaseaddress(imagebuffer,0); 

where tempaddress pixelbuffer, , videooutput instance of avplayeritemvideooutput.

for audio, use:

avmutableaudiomixinputparameters *inputparams = [avmutableaudiomixinputparameters audiomixinputparameterswithtrack:audiotrack];  // create processing tap input parameters mtaudioprocessingtapcallbacks callbacks;  callbacks.version = kmtaudioprocessingtapcallbacksversion_0; callbacks.clientinfo = (__bridge void *)(self); callbacks.init = init; callbacks.prepare = prepare; callbacks.process = process; callbacks.unprepare = unprepare; callbacks.finalize = finalize;  mtaudioprocessingtapref tap; osstatus err = mtaudioprocessingtapcreate(kcfallocatordefault, &callbacks,                                           kmtaudioprocessingtapcreationflag_posteffects, &tap); if (err || !tap) {     nslog(@"unable create audio processing tap");     return; }  inputparams.audiotapprocessor = tap;  // create new avaudiomix , assign our avplayeritem avmutableaudiomix *audiomix = [avmutableaudiomix audiomix]; audiomix.inputparameters = @[inputparams]; playeritem.audiomix = audiomix; 

regards, niek

here solution:

this implement airplay, use code audio on app don't know if can improve video can try ;)

on appdelegate.m :

- (bool)application:(uiapplication *)application didfinishlaunchingwithoptions:(nsdictionary *)launchoptions {      [radstyle applystyle];     [radiosound superclass];     [self downloadzip];      nserror *sessionerror = nil;     [[avaudiosession sharedinstance] setdelegate:self];     [[avaudiosession sharedinstance] setcategory:avaudiosessioncategoryplayandrecord error:&sessionerror];     [[avaudiosession sharedinstance] setactive:yes error:nil];      uint32 sessioncategory = kaudiosessioncategory_mediaplayback;     audiosessionsetproperty(kaudiosessionproperty_audiocategory, sizeof(sessioncategory), &sessioncategory);      uint32 audiorouteoverride = kaudiosessionoverrideaudioroute_speaker;     audiosessionsetproperty (kaudiosessionproperty_overrideaudioroute,sizeof (audiorouteoverride),&audiorouteoverride);      [[uiapplication sharedapplication] beginreceivingremotecontrolevents]; } 

an if use airplay in nice implement lockscreen control, artwork, stop/play, title ecc.

in detailviewcontroller of player use code:

- (bool)canbecomefirstresponder {      return yes; } - (void)viewdidappear:(bool)animated {      [super viewdidappear:animated];     [[uiapplication sharedapplication] beginreceivingremotecontrolevents];     [self becomefirstresponder];      nsdata* imagedata = [[nsdata alloc] initwithcontentsofurl:[nsurl urlwithstring: (self.saved)[@"image"]]];      if (imagedata == nil){      mpnowplayinginfocenter *infocenter = [mpnowplayinginfocenter defaultcenter];     mpmediaitemartwork *albumart = [[mpmediaitemartwork alloc] initwithimage:[uiimage imagenamed:@"lockscreen.png"]];      infocenter.nowplayinginfo = @{mpmediaitempropertytitle: saved[@"web"],mpmediaitempropertyartist: saved[@"title"], mpmediaitempropertyartwork:albumart};      } else {          mpnowplayinginfocenter *infocenter = [mpnowplayinginfocenter defaultcenter];         mpmediaitemartwork *albumart = [[mpmediaitemartwork alloc] initwithimage:[uiimage imagewithdata:imagedata]];          infocenter.nowplayinginfo = @{mpmediaitempropertytitle: saved[@"link"],mpmediaitempropertyartist: saved[@"title"], mpmediaitempropertyartwork:albumart};      }  } 

hope code can ;)