AI and Responsibility: Securing SAP's Digital Core

00:00:00: The biggest opportunity is basically the scale factor.

00:00:02: That's from my perspective, the biggest opportunity AI carries for every discipline right?

00:00:10: Business, cyber development.

00:00:14: we're seeing also that with AI generating coding so it's basically a scale factor.

00:00:28: It's great to have you on the show where we tackle SAP security together and looking forward to our topic today, discussing AI responsibility.

00:00:37: And of course The Digital Core in terms of securing ERP or SAP In this age Of kind of intelligent systems.

00:00:45: but before We dive-in go across different topics For listeners who might not know You Who is Jose?

00:00:54: Your journey in then out of SAP.

00:00:57: You worked with SAP for a very long time, different roles and now at PwC.

00:01:02: So maybe a little bit about you Jose?

00:01:04: Super, Waseem thanks for having me here.

00:01:07: Hello to all of our audience.

00:01:09: And yeah well my name is Jose Marquez.

00:01:12: I'm coming from an in-year history.

00:01:16: I work at SAP.

00:01:17: I started out their headquarters in Bogdor.

00:01:26: Yeah, just work my way into other roles.

00:01:31: Senior developer consultant I was architect in different aspects and parts of the organization and portfolio.

00:01:47: At some point i got interested into cybersecurity as security topics a security researcher at the SAP Security Research Labs in Sofianti Police.

00:02:02: And yeah, we worked... I had to work on several patents that are still relevant today and it was kind of like the official start for my career into cyber while then i dived into other topics.

00:02:22: when you know how is in cyber?

00:02:25: Absolutely.

00:02:27: You go first into generalist, then you specialize in things that become more to the general aspects and then do specialized again?

00:02:36: I was working with customer such as a French state where we were working or their advanced public safety and security program.

00:02:46: And yeah i'm also of European whitehack hacker collective.

00:02:56: Right after SAP, I was so much into cyber already that I was the CISO and DPO of a telco company for Morgan Stanley.

00:03:08: And then went full on to what it means to be responsible for cybersecurity security data protection having basically the full responsibility of it.

00:03:20: And yeah, currently I'm the chief AI officer of the SAP practice at PwC US, the SAP Practice of The U.S.

00:03:29: Companies.

00:03:29: and well even though i am not focused into cyber per se today you know that topic is basically permeates every aspect of our life.

00:03:43: so...

00:03:45: Absolutely!

00:03:47: You went from cybersecurity or let's go back a little bit, developing up to cyber security and now Gen AI.

00:03:54: And everything that revolves around AI.

00:03:58: but do you still touch base?

00:04:00: A little bit when you are with your customers around security of AI in general mainly on enabling them in terms of business processes or enabling AI within those customers?

00:04:13: No, I mean we do.

00:04:15: We discuss everything right like.

00:04:17: I mean enabling is let's say the easy part not sure?

00:04:23: well definitely this isn't only what our customers look having from us.

00:04:30: they basically want to be educated all aspects.

00:04:33: And

00:04:37: well, I mean the security topic in SAP and around SAP is becoming one more top of mind.

00:04:50: Ever since we have AI it began a little bit with IoT back into day like five or six years ago when you were trying to connect short-floor systems with SAP directly, right?

00:05:04: And it was this kind of like a new type of system reaching out to SAP.

00:05:10: But now is basically... I mean, with AI and with AI agents you're basically having nonhuman identities acting as users' rights.

00:05:28: That's basically the main point.

00:05:30: And in terms of security, I mean, basically SAP Security is a combination of code privilege access management or well, Access Governance Identity Management Infrastructure.

00:05:44: so it's basically when you talk about SAP Security you shouldn't understand only as managing identity or privileged access to SAP.

00:05:57: one aspect of many SAP security and an SAP from a security standpoint is basically.

00:06:04: A combination of identity access, application server security comprising also coding infrastructure because it reaches out to databases and beyond.

00:06:14: And well It has its own login monitoring right?

00:06:18: Yeah!

00:06:20: Not forgetting about data protection.

00:06:22: so yeah that's those are the aspects.

00:06:26: sometimes they they do not reach to the surface or don't have so much visibility when talking about ACP security.

00:06:34: And I mean, I caught a few areas like specifically when you are talking around SAP.

00:06:39: So maybe to dive in from speaking about AI around SAP going into those different governance.

00:06:46: but generally right now we see this kind of increase off integrating AI into business workflows.

00:06:53: and two decision-making as you said acts as human in making these decisions may be taking one step back before the business processes and workflows, how they can work from a cybersecurity perspective.

00:07:07: And in your opinion, How does this change?

00:07:10: The attack surface around these business critical systems like SAP for instance because now we're talking around the SAP not specifically in SAP as you mentioned previously.

00:07:22: so...How Does This Change The Attack Surface from that perspective?

00:07:26: Yeah, sure.

00:07:27: Basically I would say the attack surface becomes more visible.

00:07:35: for sure it's been enhanced because at the end of day AI agents are again a combination of code and prevention of scope.

00:07:45: right.

00:07:47: well they're new artifacts any type of artifact being deployed on SAP systems And that's already to say, an extension of the enhancement of their attack surface.

00:08:01: I would say in my very humble opinion... The deployment of AI and AI agents basically makes it more obvious they're already available or they've already pressed the attack surface.

00:08:18: Why?

00:08:18: Because for decades as if he was mortally The system of execution humans made decisions, SAP executed them.

00:08:26: Today we're entering in a different era one where AI systems are not just advising but act right?

00:08:34: I mean before EJTKI were having ML and all these advanced analytics you know it was like preventing and predictive.

00:08:44: so at the end of the day that decision of zero line on the end user, and we could actually make that user accountable for everything he did.

00:08:56: Now this is where the idea of a JTKI comes into play and they basically can take goals action execute tasks across in and across systems.

00:09:09: instead of human loyal to an SAP system.

00:09:12: let's say analyzing a procurement spend identifying suppliers creating a sourcing strategy or whatever, you know?

00:09:20: Yeah.

00:09:22: Is it users in SAP systems?

00:09:24: and we're basically moving away from the user taking care of every aspect of a process to digital augmentations.

00:09:34: first reaching right like everybody knows about the vision for digital co-work.

00:09:39: right And yeah well I mean at the beginning It will be almost everything kind of having a semi-autonomous implementation, right?

00:09:50: When you were still having some human interloop intervention.

00:09:55: Right not leaving everything fully to the agents.

00:09:59: but well I mean this is just based on or facing between for sure lots people out there and lots of SAP clients would love too.

00:10:13: basically automates more and more, yeah.

00:10:19: Basically too to have a more autonomous ERP an agenda ERP.

00:10:25: absolutely I mean this is something that we now see as well.

00:10:28: i mean there's these usually posts in some research.

00:10:34: i'm looking into on how possibility of AI removing even the use of ERP systems but i think we're very far from there to reach their butt.

00:10:44: I do want to hook on one thing you said where traceability or accountability is quite important.

00:10:50: so from your experience when an AI system for women agent like I recommends are even triggers a certain action that can, probably impact a business process or the critical business process, who ultimately carries this responsibility?

00:11:07: That is great question.

00:11:12: It's absolutely a great question that will be asked by internal audits and it'll ask external audit procedures.

00:11:22: are questions needed to maintain certain certificates in certain industries, if we deploy an AI agent that is not just having a human interloop at every step on which these agents commit something to the database or imposes one.

00:11:46: If you have intelligence with autonomy then basically he needs every step and every action of this agent.

00:12:03: because then all decisions, these agents made.

00:12:06: All the information or data it generated that will be influencing further processes downstream in the ERP.

00:12:19: everything needs to be locked Everything needs to traced.

00:12:21: Because basically first agents need just taking the analogy to humans.

00:12:31: that will need to be audited and made accountable for your actions.

00:12:36: And then, for sure if there was something wrong with some wrong decisions were taken than you'll make it.

00:12:42: look into the agent... Into configuration of agents let's say.. Into knowledge phase version.. Into system prompt.. Into integrations.. Into configurations of agents To Look Okay what happened?

00:12:58: To this agent say like it's something wrong and you were really sure about the knowledge base.

00:13:06: But whilst that system prompt got updated or I don't know some rules guardrails gods updated are outdated, right?

00:13:18: And then you have drive so you should be able to track trace an audit basically everything at That's that's an agentic decision otherwise you might risk I mean besides Besides let's say some bad decisions, right?

00:13:36: Like okay like they'd say yeah.

00:13:39: I mean there were some reverberable positions But you can deal with them as clients but if your not able to audit the behavior of Your own agents then you all start having some troubles when An external audit comes around the corner or when you have to comply with certain certificates.

00:14:03: and imagine that your system will act as a black box from an audit perspective.

00:14:15: not so ideal.

00:14:16: No, I see the point i definitely do but does bring back a question in my head whether it's two-part questions.

00:14:23: from that perspective and just looking into history especially around historically with implementation deployments around target SAP right now when it is specifically around SAP security always lagged.

00:14:38: It was great ERP software And now with AI, a lot of organizations have this some sort of strategy.

00:14:49: I would say AI strategy but security governance kind of lags behind.

00:14:55: Now the two-part question is or yeah i would say to part questions first Of all what are these common gaps they see when you Have this kind of high ambition and low AI Governance?

00:15:06: that's from an organization point of view And based on what you said, are regulators and even audit companies ready or have the means to actually start auditing these agent tech AIs?

00:15:20: Based on this kind of deployments.

00:15:22: Or based on these responsibilities and requirements that businesses put right now towards its AI

00:15:28: strategy?".

00:15:29: Yeah I mean before... yeah so it's also a great follow-up question.

00:15:36: Before Auditing Or being able to audit agents I mean like false needs need to be able to capture the right metadata.

00:15:45: Yeah And so far, I would say a genetic AI agents are being treated as I mean Like from a. from a technical standpoint.

00:15:56: for my point they have been treated as a regular pieces of softwares by yeah quite an engine.

00:16:04: you basically, yeah just press play and you'll get it to go.

00:16:10: You monitor here on there right?

00:16:12: There are.

00:16:12: no I would say there are no standards in monitoring agents such as that...you know..there are no comprehensive standards in Monitoring Human Employees

00:16:23: Right?

00:16:25: Exactly!

00:16:25: And if both are having or will if the objectiveness at some point in time, AI agents will become as autonomous as a human in this system.

00:16:40: Then there is no template actually to monitor them fully and to audit it fully as we say.

00:16:47: where are humans involved?

00:16:49: Even from very beginning they're non-human identities that would be taken over more tasks of humanity right and they will have let's say a feature, feature complete this free feature equality among the human agents or like yeah a human in an AI agent.

00:17:12: Let's see it some scenario right?

00:17:14: And I'll tell you that at some point an agent then a human need to exactly the same order able to do extract of the same thing in the system.

00:17:23: so before we were not Able to or we were able to audit the human by asking him, right?

00:17:33: Okay.

00:17:33: How did you?

00:17:35: I took this decision y'all I got an email and And actually it was an emergency he was I don't know.

00:17:42: so what is a fire drill where they have To do something quick?

00:17:45: okay well are you happy in your?

00:17:47: yeah sure ok all you have.

00:17:49: Ok cool super let's say that's The old good.

00:17:53: next question right This is this basically how we audit humans right now and how do we want to or are going to be able to audit?

00:18:09: That's a big question.

00:18:11: Are you going ask them questions, such as if they were the very humans that might be automating their activities from Probably more or less, with less interaction.

00:18:31: That's a big question.

00:18:34: What we know is that from a compliant standpoint and an audit standpoint Clients in the whole industry will need to evolve.

00:18:52: We can probably try And I imagine, right?

00:18:55: Like in a couple of years from now asking the AI agent.

00:18:59: The question why the AI did something or didn't do something and that will be fine.

00:19:06: but probably five years from those questions we'll already coming with... ...the log trail.

00:19:18: as the agent does its stuff.

00:19:22: You speak to Leaders or cybersecurity leaders that are trying to position a I towards their strategy.

00:19:30: How would you at least recommend?

00:19:33: Or how should those leaders discuss this inside the enterprise?

00:19:38: like there are definitely areas to put around agentic AI or depending on the adoption that is being carried out within the organization, what kind of discussions should start right now?

00:19:53: Or how should leaders discuss this?

00:19:55: and maybe on a side note are they discussing it.

00:19:58: But How Should They Discuss This Kind Of Governance Accountability Guardrails That Need To Be Happening Inside The Enterprise?

00:20:09: That's mostly my starting topic.

00:20:13: My clients and it basically started with a risk management question, right?

00:20:16: Like I mean you would like to automate this process or everybody is automating And you want to automate as well.

00:20:25: cool so it starts.

00:20:27: It starts all over the risk management questions.

00:20:29: Okay So do You Want To Automate This Process And Exactly This?

00:20:34: The Steps At this in these points, so how does your risk management or appetites look like?

00:20:41: I need to feed you?

00:20:43: You say you can deal but you have so much volume of Something in process and new basically in your team kind of deal cannot possibly deal with With all the ballroom off.

00:20:56: Let's see uh i don't know orders production order sort of something of a sort.

00:21:03: then then you say, okay look if it's better that I process something at least try to automate.

00:21:11: Let's see your take.

00:21:12: yeah i have some mistakes there but in order do that than better to leave money on the table or to leave things unprocessed?

00:21:24: Or maybe... If I'm not processing these quick enough basically according penalties from my contracts or all related things.

00:21:36: And then, and only then is when together with the client we agree that it's actually better to engage into a genetic and do something about it than not do anything.

00:21:55: Once we agree, okay well it's not good to acquire penalties or be subject to other things.

00:22:03: Or just to leave money on the table and there are not enough people with the right skill sets to bear them out of work.

00:22:13: Okay cool then let's go after this And deploy some agent workflows to support it.

00:22:21: So that will basically be a risk.

00:22:25: Yeah, risk management will be basically the first aspect.

00:22:33: Risk management is a very loaded topic and subject that can open up so many days.

00:22:38: but Is there any kind of frameworks?

00:22:41: I know you mentioned earlier That isn't this standard.

00:22:44: But are there any semi-frameworks or principles that organizations Can adopt right now?

00:22:53: I mean, there are some... Here on this side of the Atlantic in the underwear of the UAI Act.

00:23:10: That one is not so relevant.

00:23:13: here at the Americas We basically kind-of go with standard frameworks like NIST, ISO twins in terms of One and well try that everything we do development wise, deployment-wise and scope-wise is basically answering already these questions.

00:23:33: That would not deviate from the information security management system on the ISMS than the clients are already having.

00:23:42: On top of that We also like a lot to apply to every relevant meaning automated, agentic implementation.

00:23:58: We would like to apply the top ten vulnerabilities that have been worked out by all of us.

00:24:08: That's

00:24:10: a great resource.

00:24:11: there are always good guidance.

00:24:13: I think they just got updated and mapped against Other frameworks that already exists.

00:24:22: so then.

00:24:22: That's also great Guidance.

00:24:24: also prevent situations even if you did something or everything right, you know like on the paper following The standards are the controls your is and s it's only having.

00:24:40: It's always a good idea to apply.

00:24:42: there was top ten below abilities just to be honest.

00:24:48: Yeah, it is quite a extensive resource and really a lot of work has been done around there.

00:24:54: I did see that NIST has released some sort of standard.

00:24:58: no its not a standard but maybe a framework or regulation haven't dug deep into it.

00:25:05: yeah didn't dig deep too much.

00:25:07: i do believe they're still alot to adopt.

00:25:09: when to these frameworks around agent

00:25:12: A.I.,

00:25:13: or the adoption of AI?

00:25:14: Maybe if we... I want zoom a little bit on cyber security, so how it is reshaping Or AI's reshapping cybersecurity From your point-of-view where do you see biggest impact in security operations Detection Investigations Automation Or completely something else?

00:25:32: when comes to AI making this kind big impact there?

00:25:37: In terms of security operations and AI From my very humble point of view, the impact should be in operations.

00:25:45: Why?

00:25:46: Because basically we're going to be attacked by malicious agents leveraging AI or it's also getting automated on the other side.

00:25:58: right you don't need... Basically will be a need for less human labor to perform attacks whatever the malicious side will be performing.

00:26:11: And we just had the news, right?

00:26:16: It was one of these two famous selling companies and one of them got used to hack a government that hacked into other things being made so it would do a red teaming exercise.

00:26:33: You know, it was not even kind of like a state-sponsored or whatever.

00:26:37: It would just be somewhat right.

00:26:40: so

00:26:40: absolutely

00:26:41: and So imagine now the power that That organized actors are state sponsors.

00:26:49: Actors aren't going to be having in months from them Right?

00:26:53: Yeah So I say set the impact that AI should be having, I am actually not aware.

00:27:07: Where the impact is now?

00:27:10: The impact should be geared towards operations because this where a security operation center or playbooks being run and from detection to orchestration of actions in playbooks should actually be happening.

00:27:30: This is where I see, or what I hope that we integrate AI as well?

00:27:35: Yeah!

00:27:37: You said it correctly... AI is being used by both attackers and defenders.

00:27:43: even if you go to consulting companies We are testing with different stuff around AI To improve our capabilities As well as our defense capabilities a higher level, like management perspective.

00:27:59: If we are on the same topic when it comes to security decisions is there a risk that organizations begin to completely rely or too heavily rely on AI-driven security decision?

00:28:13: What's your thoughts about

00:28:13: this?

00:28:14: Well I mean... There're always risks of you know in life in general in terms of yeah overdoing right General Risk Of Life even if it applies to your investment, don't put all the Xs in one basket.

00:28:38: AI should allow security organizations to scale because this is going to be a big thing.

00:28:48: so far our scale attacks were geared towards a large organizations, right?

00:28:57: Where it won the world where they were.

00:29:00: The offside was big enough to invest such amount of resources.

00:29:05: now you just need two.

00:29:08: basically get an AI tool to do the job for you and You don't even need to be

00:29:16: paid

00:29:16: any like with any known malicious actor or state funds from whatever.

00:29:21: so imagine this recent attack would have had the instruction of like, yeah don't only do this to that.

00:29:29: To these but do a red teaming exercise so every discoverable IP address in the internet.

00:29:36: So from that will be still running right?

00:29:39: And well... The point is I want make it's at least in terms human labor and profitable.

00:29:55: logistics will be so reduced that now large-scale unpervasive attacks, we'll be geared to basically everybody.

00:30:08: Everybody even at an individual level.

00:30:11: there is actually... I mean logistically.

00:30:14: for sure you need some money but besides that Give it a range of my piece.

00:30:24: or yeah, right?

00:30:27: Does

00:30:27: the attack

00:30:28: like that?

00:30:29: they just just do a rectum in exercise see the country off you know.

00:30:36: And so and and that's going to be the problem.

00:30:40: Well, that's good situation That attacks will be basically just getting started on the glue and by and by virtually anybody.

00:30:50: So we actually need and all security layers needs to basically gear up for that, right?

00:30:59: From let's say from infrastructure security providers.

00:31:02: To application security providers.

00:31:04: two consultancies too Basically everybody from layer one to layer seven even to layer eight Right speaking about the user so everybody to get ready for this wave.

00:31:21: that I mean, i don't hope but see it.

00:31:24: Yeah

00:31:25: you mentioned put your eggs in one basket.

00:31:28: quite an investment strategy But also a strategy and life now.

00:31:33: with that being said we spoke about the AI role.

00:31:36: what role should human analysts then continue to play?

00:31:41: In this hybrid of AI assisted security environments

00:31:45: they will need They need to fulfill an oversight role.

00:31:54: Basically they'll need to embody a manager, basically the manager of that team right?

00:32:00: Yeah and basically I'm managing managers because maybe this person is.

00:32:10: individuals having these roles will not be able to fall off on every decision being made by the agents being deployed, or automations being deployed into their landscape.

00:32:25: Or in to their field of responsibility.

00:32:28: but they will be having agents checking upon every decision been made.

00:32:32: Checking on if everybody is having same knowledge base.

00:32:36: Checkin' If there wasn't a tool hijack A misuse or exploitation If the agents are still running according to their privilege access, if there is nothing unexpected or communicated and infiltrated right?

00:32:57: Or maybe some agents didn't go wrong because of brown configurations.

00:33:07: So all these things can be monitored and reported to the human else.

00:33:17: Humans managing agents across their environments, which... Human

00:33:22: Managing Agents?

00:33:23: Managing agents!

00:33:25: Yes humans managing.

00:33:27: they're managing agents but let's try to tie everything I want it tried to tie Everything together.

00:33:33: i mean It is not of course just AI Not a topic only affecting core business applications or at least The Digital Core.

00:33:41: But this Is where A lot Of Critical information out of critical data exists at the heart of these digital core or ERP systems.

00:33:50: So when we are, at least coming back to SAP environments where they are a part many organizations from your perspective how should leaders maybe think about this?

00:34:02: AI risks because also SAP is introducing or introduced several capabilities now and Pushing towards agentic AI, how should leaders think about these risks?

00:34:12: AI risk specifically when protecting their digital core.

00:34:16: Yeah I mean as I just said previously it's all a risk perspective.

00:34:25: if the risk is bearable then why not right?

00:34:29: and SAP is actually having the SAP jewel that can be activated along?

00:34:39: your ACP transaction is actually a great way to start the AI or the AI agent journey specifically for an ACP ERP.

00:34:50: So, so th-that's a great ways of starting this stool.

00:34:58: agents are basically embedded into your transactions So it basically leaves inside the transaction.

00:35:12: It can access its knowledge base, limited to a knowledge based of the transaction or application that's actually deployed at.

00:35:21: so makes you quite control start into topic your end users get mirrorized with these new features and And before you go further, right?

00:35:39: I mean that there will be for sure.

00:35:41: I think the Apple Pro released agent builder and well on BTP You can basically do whatever we want.

00:35:49: That's your playground.

00:35:50: pass of SAP.

00:35:52: so actually to all the full stack development you wanna do but in terms like getting forward dual It's actually a great way to do it.

00:36:07: Okay, so I believe there is definitely still...I had look into and around observability at least around operations as you mentioned earlier.

00:36:20: of course maturing this yes great but its still maturing let say hold this kind accountability or have somebody accountable definitely requires for customers or people listening to take care when adopting.

00:36:38: It's a great adoption, and also from our perspective how we are doing things faster working in an innovative way.

00:36:53: I could say that because of we are deploying these different stuff to also support in what we provide to customers, not just from a security perspective but kind of understanding quite quickly the environment.

00:37:07: What is going on?

00:37:09: And i do believe within business process this definitely something of great value Before we start closing and usually like ask questions To leave something for the audience, something memorable.

00:37:23: For the audience to take with them.

00:37:25: from your experience and now you're working a lot With major companies in this topic From your opinion What is the biggest opportunity?

00:37:34: And The biggest risk AI introduces for organizations?

00:37:39: The biggest opportunity Is basically the scale factor.

00:37:42: that it is.

00:37:43: That's from my perspective.

00:37:48: Discipline, right?

00:37:49: Business cyber development.

00:37:52: We're seeing also that with AI generating coding.

00:37:56: So it's basically the scale factor.

00:37:58: but the biggest risk is that from my perspective The biggest risk Is that humanity itself starts to get too comfortable To that and That folks stop reading or writing right through email and just fronting whatever they want to ask.

00:38:20: And then, it's getting that email written for them or you know making the effort to go through ten pages of PDF by just uploaded and asked for an executive summary.

00:38:33: I mean... That is okay when are in a hurry but don't forget how to read your book.

00:38:42: Don't forget how you fall down to a yes don't forget all that right.

00:38:48: because then maybe in the future when enough people for get about how to read actually something from beginning to an end.

00:38:58: Reading books without getting distracted with and we remembering everything was written on book having opinion after that probably is kill dad will be scars in the gym.

00:39:15: I like that, because it reminded me of a post i saw on this newsletter or newspaper released one of their edits and there was at the end of one of the columns the prompt from chatty pt what tells you should do something else for you?

00:39:33: Or try to summarize so... There has been up to this point a few stories out there where auditors or consultants even, to bring it that part of providing support and consultancy to customers without reading an actual outcome what they are producing.

00:39:56: It may... Of course there were a few story's out there.

00:39:59: our listeners can read too but I do agree on the final points from your side Jose.

00:40:05: Jose thank you very much for Joining the SAP Cyber Security by No Monkey, it's great to have you.

00:40:11: Great always speak about these topics.

00:40:13: I mean there is a lot of areas that we usually touch base on offline but i enjoyed our talk and We definitely would have your back for different Areas to Talk To You Soon.

00:40:24: Super was him.

00:40:25: Thankyou For Having Me.

00:40:32: Thanks to all.

00:40:39: No, it was not.

00:40:41: Thank you Jose and I will see you soon!

00:40:43: Thanks Jose, take care.