Last updated on November 4, 2022.
- November 2022: Demonstrated a more elegant solution using
N/query
instead ofN/search
. Refer to this section for more details.
TL;DR
Custom Lists are often used in NetSuite customization endeavors. Manipulating custom list values via scripting requires referencing them by their internal IDs which are environment-specific. This could lead to subtle bugs and increase deployment overhead. By using the environment-agnostic script ID instead, we can achieve the desired results without worrying about internal ID inconsistencies. There’s a freebie included which you can grab here.
Context
NetSuite offers the possibility of creating custom lists that can be used to provide a dropdown list of pre-defined options on other (custom) records. Custom lists are an essential element in NetSuite’s powerful customization arsenal and facilitate modeling the “real world” in NetSuite. For example, say, we want to capture a risk management process in NetSuite. We will likely create a Risk Assessment
custom record. One of the fields on that record may be a Risk Level
which we conveniently can capture using a custom list with values such as “Low”, “Medium” and “High”.
Custom Lists are accessible via Workflows and SuiteScript, making them developer-friendly. However, there is a significant limitation that causes them easily become a developer’s nightmare.
Problem
Following up on our Risk Assessment example. Suppose, I want to create a new Risk Assessment entry via script and set the risk level accordingly, the code will look something like this:
var riskRec = record.create({
type: 'customrecord_nsi_risk_analysis'
});
// TODO: Determine risk level based on business logic.
riskRec.setValue({
fieldId: 'custrecord_nsi_risk_level',
value: '1' // <- This is the internal ID of the corresponding value from the risk level custom list
});
// Set other fields and save
riskRec.save();
The issue is that to set a custom list value as illustrated in line 9 of the snippet above, we need to use the internal ID of the value. However, as you may know, internal IDs are environment-specific. This means that, for the same custom list value, the internal ID in Production might be different from that in Sandbox.
If it is not obvious to you how this situation could quickly become problematic, here are two examples of how custom list internal IDs can diverge across environments and the consequences thereof.
Problem Scenario 1: Updating Existing Custom Lists Can Cause Divergence of Internal IDs
Suppose Larry, our lead developer at Asoville Inc., is working on the risk management feature in Sandbox. He realizes that he needs to add more options to the Risk Assessment Status
custom list. He goes ahead and makes this change in Sandbox. However, when reproducing the change in Production, NetSuite assigns different internal IDs as indicated above. As a result, the code which he implemented and validated in Sandbox will now behave differently in Production. Specifically, attempting to set the status “Done” using internal ID “3” fail in Production with an “INVALID_FLD_VALUE” error because that internal ID is not assigned to any list value in Production. More subtly, attempting to set the status to “Blocked” using internal ID “4” will work fine in Sandbox but will silently result in the status being set to “Done” instead in Production! Ask any developer and they’ll tell you how painful these kinds of bugs are to track down.
While the scenario sketched above typically will not happen if Larry creates the new values in the same order as they were created in Sandbox, that’s not a guarantee. NetSuite does not provide any semantics on internal IDs beyond the fact that they are guaranteed to be unique integers within a given environment. From experience, we know that they are generally non-negative and increasing. However, in some cases, they might be sequential, whereas in other cases, they might jump by a few hundred at a time e.g. 1, 101, 201, etc. In this specific example, the skipping of internal ID “3” in Production could be because a value with that internal ID previously existed but got deleted. The next entry will not reuse that ID but skip to “4” instead.
Let’s look at an even more subtle scenario.
Problem Scenario 2: Deploying New Lists via SDF Can Cause Divergence of Internal IDs
Larry is a progressive NetSuite developer. So he uses the SuiteCloud Development Framework (SDF) [I]NetSuite (September 2, 2016). SuiteCloud Development Framework Overview. Available at: https://netsuite.custhelp.com/app/answers/detail/a_id/51622. [Accessed August 28, 2020] during his development and for deploying changes from Sandbox to Production. He’s modeled the client’s risk management process for which he created a new Risk Level
list in Sandbox. When deploying via SDF, he simply assumes that the order of the internal IDs will be the same as in Sandbox. After all, it is a new list, right? Very wrong! The outcome is depicted below: The internal IDs are as different as night from day.
The reason for this discrepancy is rooted in how SDF works. When SDF creates a new custom list, it creates the values in the order in which they appear in the XML definition of that list without any regard for their internal IDs. This will typically correspond to the order in which they are displayed in NetSuite. In our example, Larry went through an iterative process as most developers do. Along the line, he rearranged values in the Risk Level
custom list in Sandbox. Interestingly, when SDF replicated this list in Production, NetSuite ignored the existing internal IDs (which are Sandbox-specific) and instead assigned sequential internal IDs to the custom list values. The outcome is as devastating as in the previous example but slightly harder to spot.
Note that I specifically call out SDF here because, from my tests, if the same (new) custom list were to be deployed using SuiteBundler [II]NetSuite (March 4, 2020). SuiteBundler Overview. Available at: https://netsuite.custhelp.com/app/answers/detail/a_id/8082. [Accessed September 28, 2020)] – a different deployment technology offered by NetSuite – the order of internal IDs will be preserved. From my experience, having worked quite extensively with both SDF and SuiteBundler, SDF is newer, more developer-friendly but also less “intelligent” than SuiteBundler.
A detailed analysis of these two deployment technologies is beyond the current scope. The key point is that your code should not depend on the (unpublished) details of how NetSuite generates internal IDs. (They are called “Internal” for a reason.) Instead, you should be sure to avoid problems arising from internal ID mismatches. Let’s look at a few ways to do that.
Common Solutions
Let me start with the typical solutions you’ll find out there before showing you a more robust approach.
Common Solution 1: Ensure That Internal IDs Always Align Across Environments
This is a proactive approach to prevent divergence from occurring in the first place and attempting to fix it at the custom list level if it does occur. A few ways of ensuring “alignment” of internal IDs are:
- Always create your custom list values in the same order in all environments as this will generally produce the same sequence of internal IDs.
- When faced with the “skipping” internal ID situation described earlier, reproduce it in the other environment(s) by deleting entries as needed to force a skip there as well.
In reality, the above steps are easier said than done. Nevertheless, a seasoned NetSuite developer or administrator is able to achieve alignment in most cases. However, this approach remains, at its best, grasping at straws and will sooner or later prove unsustainable.
Common Solution 2: Track & Fix Any Divergence Immediately After Deployment By Updating the Code in the Target Environment
const RiskLevel = {
TO_BE_DETERMINED: '6', // In Production: '1'
EXTREMELY_LOW: '9', // In Production: '2'
LOW: '1', // In Production: '3'
MEDIUM: '2', // In Production: '4'
HIGH: '4', // In Production: '5'
EXTREMELY_HIGH: '10', // In Production: '6'
ULTRA_HIGH: '5', // In Production: '7'
DANGEROUS: '7', // In Production: '8'
UNIMAGINABLE: '8' // In Production: '9'
}
// TODO: After the next SB refresh, update to match values from Prod.
/**
* Definition of the script trigger point.
*/
function execute(scriptContext) {
var riskRec = record.create({
type: 'customrecord_nsi_risk_analysis'
});
riskRec.setValue({
fieldId: 'custrecord_nsi_risk_level',
value: RiskLevel.TO_BE_DETERMINED
});
riskRec.save();
}
If the previous approach does not work or is not feasible, the NetSuite team typically resorts to a reactive approach in which the divergence is identified and noted in the changelog. As part of the post-deployment steps, the internal IDs used in the code are changed to match those in Production and we’re all good… right? Well, kind of…
Obviously, changing code post-deployment introduces the risk of breaking something that has previously been validated. Also, this approach essentially displaces the problem instead of solving it. It introduces a divergence in code which typically gets fixed during the next Sandbox refresh at which point the internal IDs will again naturally align.
I’ve seen this go terribly wrong where, after deployment, a hotfix needs to be done and the updated code is accidentally overwritten in Production. On the other hand, I understand developers resorting to this solution in the absence of a better alternative. Well, gladly, there is a little known and, arguably, neater alternative as I discuss next.
A Better Way
The solution described in this section can be summarized in one sentence: Use script IDs to reference custom list values instead of internal IDs because, unlike internal IDs, script IDs are environment-agnostic. Let me walk you step-by-step through the process.
Step 1: Enable Displaying of Script IDs on Custom Lists
Script IDs are available records like workflows, saved searches, roles, custom records, and custom lists. On custom lists, in addition to the list having its script ID, each value also has a script ID. Note that, to see the script IDs of custom list values, you need to enable the Show ID Field on Sublists
preference as illustrated below.
Step 2: Assign a Unique, Meaningful Script ID to Each Custom List Value
NetSuite generates a default script ID for each value when the custom list is saved. As per best practices when working with script IDs in general, adjust the script IDs to make them more meaningful. I recommend you maintain NetSuite’s convention and start your list value script IDs with “val_”. (Note that unlike other records, NetSuite does not automatically include this prefix for you.)
Obviously, make sure that the script IDs you define for a custom list values in Sandbox match those in Production. Note that if you create a new custom list and deploy it via SDF or SuiteBundler, whatever script IDs you defines are propagated to Production when you deploy.
Going back to our working example, notice that although the internal IDs are different, we’ve ensured that the script IDs match. With this in place, the next step is to write some code that takes a script ID as input and returns the internal ID.
Step 3: Retrieve the Required Internal IDs Using Script IDs
Custom lists can be searched via script [III]NS Developer (October 1, 2016). Accessing Custom List Using Scripting. Available at: https://netsuiteusersupport.blogspot.com/2016/10/accessing-custom-list-using-scripting.html. [Accessed September … Continue reading and accessed via N/query
. And that, my friend, opens a door to a more reliable solution. Instead of directly working with internal IDs, we can write a function like the one below that uses the environment-agnostic script ID to retrieves the internal ID.
Tip: Note that, earlier versions of this article, we demostrate a more complicated solution using
N/search
. If you implemented that solution, we highly recommend that you switch to theN/query
-based approach demonstrated below as it is much easier and more elegant.
Here’s the N/search
-based approach
/*
* Retrieves the internal ID for the custom list value
* with the specified script ID using N/search.
*/
function getListValueId(list, listValueScriptId) {
var listValueId = null;
search.create({
type: list,
columns: [
'internalid',
'scriptid'
]
}).run().each(function(result) {
// Note: From tests, the script ID from the search results are always uppercase.
if (result.getValue("scriptid") === listValueScriptId.toUpperCase()) {
listValueId = result.getValue('internalid');
}
// Stop iterating when we find the target ID.
return (listValueId === null);
});
return listValueId;
}
Here’s the N/query
-based approach. Notice that we don’t have to loop which makes it more compact.
/*
* Retrieves the internal ID for the custom list value
* with the specified script ID using N/query.
*/
function getListValueId(listType, listValueScriptId) {
var listValueId;
const sql = "SELECT ID FROM ? WHERE scriptid = ?";
var resultSet = query.runSuiteQL({ query: sql, params: [listType, listValueScriptId.toUpperCase()] }).asMappedResults();
return listValueId = (resultSet.length === 1 ? resultSet[0].id : null);
}
Note that the above implementation is very naïve and will not be governance friendly. I’ve presented it just to illustrate the concept; I do NOT recommend this approach in production code. Instead, use the CustomListLookup
class which you can get here for free. It runs the search/query once for a given list, resulting in a fixed 10 governance units overhead per list regardless of the number of values in the list. In the next section, I illustrate this approach.
Tying It All Together
Returning to our example, here’s a complete implementation for our Risk Level
custom list. This code, when run in Sandbox or Production, will set the correct Risk Level
value, even if the internal IDs in both environments are different.
define(['N/record', '/SuiteScripts/netsuite-insights.com/utils/NSI_CM_CommonUtils'],
function(record, nsiUtils) {
const RiskLevel = {};
// NetSuite does not allow us run any code that accesses its APIs
// in a define context, so we need to initialize after the APIs are
// loaded and ready, i.e., in the return section.
function initialize() {
const riskLevelLookup = new nsiUtils.CustomListLookup('customlist_nsi_risk_level');
RiskLevel.TO_BE_DETERMINED = riskLevelLookup.getId('val_nsi_risk_level_to_be_determined');
RiskLevel.EXTREMELY_LOW = riskLevelLookup.getId('val_nsi_risk_level_extremely_low');
RiskLevel.LOW = riskLevelLookup.getId('val_nsi_risk_level_low');
RiskLevel.MEDIUM = riskLevelLookup.getId('val_nsi_risk_level_medium');
RiskLevel.HIGH = riskLevelLookup.getId('val_nsi_risk_level_high');
RiskLevel.EXTREMELY_HIGH = riskLevelLookup.getId('val_nsi_risk_level_extremely_high');
RiskLevel.ULTRA_HIGH = riskLevelLookup.getId('val_nsi_risk_level_ultra_high');
RiskLevel.DANGEROUS = riskLevelLookup.getId('val_nsi_risk_level_dangerous');
RiskLevel.UNIMAGINABLE = riskLevelLookup.getId('val_nsi_risk_level_unimaginable');
}
/*
* Definition of the script trigger point.
*/
function execute(scriptContext) {
initialize();
var riskRec = record.create({
type: 'customrecord_nsi_risk_analysis'
});
riskRec.setValue({
fieldId: 'custrecord_nsi_risk_level',
value: RiskLevel.TO_BE_DETERMINED // <- Always correct regardless of internal ID divergence.
});
riskRec.save();
}
return {
execute: execute
};
});
Let me explain some aspects of this sample code:
- Line 3:
nsiUtils
is a helper module that implements the logic for mapping script IDs to internal IDs in aCustomListLookup
helper class. You can download the module here for free (or write yours based on the reference implementation provided in Step 3 above, if you so desire). If you create your own implementation, be conscious of governance considerations. - Line 10: Each
search
execution uses at least 10 governance units. To avoid performing a search to resolve each list value’s internal ID, we use theCustomListLookup
class which loads the custom list once into memory. Subsequent calls in lines 12 – 20 utilize the cached value and incur no additional governance costs. - Lines 27: Ideally, I’d have loved to initialize the “enum” on Line 4 where it was defined but doing so will result in the following error: “All SuiteScript API Modules are unavailable while executing your define callback”. This is a known NetSuite constraint as explained here by Eric T. Grubaugh. As such, the
initialize()
function is needed and must be executed after all APIs are available since our solution uses theN/search
API. - The rest of the code is as before and requires no further explanation. Please note that this is a reference implementation to illustrate the concept; there’s likely room for further optimization.
Further Considerations
While this solution works great, it does incur some overhead. First, instead of a static lookup, we’re forced to resort to a dynamic lookup which incurs at least 10 governance units per custom list. While this is not a huge drawback, it is something to be borne in mind. Secondly, the initialize()
function is arguably a bit harder to read than the original code. Unfortunately, NetSuite’s restriction of not loading API modules in a define callback makes it difficult to optimize the code further.
In my opinion, NetSuite (not developers) should ultimately solve this problem by providing an environment-agnostic way of setting custom list values in code. I hope that sometime soon, we’ll be able to simply use the script ID in a setValue
call as illustrated below. After all, APIs like search.load()
accept both Internal and script IDs so it can be done.
var riskRec = record.create({
type: 'customrecord_nsi_risk_analysis'
});
// Determine risk level based on business logic.
riskRec.setValue({
fieldId: 'custrecord_nsi_risk_level',
value: 'val_nsi_risk_level_to_be_determined' // Hopefully, such code will someday be supported.
});
// Set other fields and save
riskRec.save();
Until then, be careful when working with custom lists, use the recommended solution whenever possible, and save yourself unnecessary headaches. And be sure to grab the CustomListLookup
helper class here to kick start things. All the best.
Wrap Up
Never stop learning! Until very recently, I only knew of and applied the two common solutions described earlier in this article. However, as I listened to fellow NetSuite professionals express their frustration about this glaring limitation of NetSuite’s custom lists, we all wondered why script IDs are not more useful. Then I came across this blog post that helped me realize that script IDs on custom lists are searchable, I did some testing and here we are with a new, more dependable solution. Perhaps someone already knew this solution but never bothered to share. In any case, now you know.
Do you have other (read: better) ideas on how to tackle the issue of custom list internal ID divergence and/or improvements to further optimize the solution(s) presented in this article? If so, drop a comment and share your thoughts. While you’re here, be sure to sign up to receive our bi-monthly NetSuite Insights like this one (and unlock any current or future access-restricted freebies). Finally, spread the word to others and show your love with claps, shares, comments, mentions, etc. And most importantly, continue to push the limits.
Further Reading
Pretty good, we do the same … however, we wrap it in the N/cache module.
Interesting… I’m cautious of N/cache as it is not clear how often it gets cleared given that it is to be “used to temporarily store data (on a short term basis)” as per the documentation.
When trying out this Utility script, I am getting the following error in NetSuite:
Fail to evaluate script: {“type”:”error.SuiteScriptModuleLoaderError”,”name”:”UNEXPECTED_ERROR”,”message”:”missing ; before statement (/SuiteScripts/Utils/NSI_CM_CommonUtils.js#26)”,”stack”:[]}
What am I doing wrong?
Thanks Wes for pointing this out. We’ve downgraded the utility to SS2.0 for maximum compatibilty. The new version will work in both SS2.0 and SS2.1 scripts.
Great Article. It’s pretty governance intensive to run a search for each list item. In your example, you’ve used 90 governance units (10 for each list item), and Suitelets, User Events, Client, and Workflow Action Scripts all have a limit of 1000. I’d love to see a solution where you run one search that pulls every list item and then create an object representing the list. Then you can map based on that. Object would look like this:
{
val_yourval1: 1,
val_yourval2: 2,
}
But overall, the idea is definitely worth noting and using. Thanks!
Thanks for dropping by Stephen! You’re right that a very naïve implementation as presented in Step 3 of the script ID-based solution will not be governance friendly. I do NOT recommend using that approach.
I think you missed the CustomListLookup class which I created and does just what you said! It reduces the utilization to a fixed 10 governance units per list regardless of the number of values. You can get the utility class here. Obviously, you can further optimize by initializing only the lists you use in a particular script instead of all your lists. Also, when a list gets stable (e.g. after the next Sandbox refresh), you can easily switch back to the static form.
I’ve updated the article to highlight the CustomListLookup class and clarify that the naïve approach is simply for illustration purposes. Thanks for the feedback.
Cheers.